title

text

D
Dmitry Melnik ИСП РАН
13:45 05 February
22 мин

Speeding up query execution in PostgreSQL using LLVM JIT compiler

Currently, PostgreSQL uses the interpreter to execute SQL-queries. This yields an overhead caused by indirect calls to handler functions and runtime checks, which could be avoided if the query were compiled into the native code "on-the-fly" (i.e. JIT-compiled): at a run time the specific table structure is known as well as data types used in the query. This is especially important for complex queries, which performance is CPU-bound. At the moment there are two major projects that implement JIT-compilation in PostgreSQL: a commercial database Vitesse DB and an open-source project PGStorm. The former uses LLVM JIT to achieve up to 8x speedup on selected TPC-H benchmarks, while the latter JIT-compiles the query using CUDA and executes it on GPU, which allows to speed up execution of specific query types by an order.

Our work is dedicated to adding support for SQL query JIT-compilation to PostgreSQL using LLVM compiler infrastructure. In the presentation we'll discuss how JIT-compilation can be used to speed up various stages of query execution in PostgreSQL, and the specifics of translating an SQL query into LLVM bitcode to achieve good performing native code. Also we'll present preliminary results for our JIT-compiler on TPC-H benchmark.

Слайды

Видео

Другие доклады

  • Дмитрий Воронин
    Дмитрий Воронин ОАО "НПО РусБИТех"
    22 мин

    The base version of DBMS PostgreSQL can register events bellow: - connect and disconnect - denial of access with date, time and user's name.

    Requirements guidelines for audit subsystem is much wider possibilities basic version of database PostgreSQL.

    The RusBITech company holds the necessary improvements PostgreSQL database to enhance it's functionality.

    A result of improvements audit subsystem of PostgreSQL database as part of the operating system «Astra Linux Special Edition» further enables registration:

    • The creation and destruction of database objects;
    • Changes to the rules of access control;
    • Both failures and successful attempts to access the database objects;
    • Changes to the powers and status of subjects access objects access.

    For all events are specified: - date and time; - The user performing the action of the registrant; - The object on which the action is carried out; - Type of event; - The result of the operation.

    Audit subsystem of modified PostgreSQL is integrated into a centralized audit system OS «Astra Linux Special Edition». Provided the rules setting without stopping the logging (restarting) database.

  • Bruce Momjian
    Bruce Momjian EnterpriseDB
    45 мин

    Postgres Going in Multiple Directions

    Postgres 9.5 adds many features designed to enhance the productivity of developers: UPSERT, CUBE, ROLLUP, JSONB functions, and PostGIS improvements. For administrators, it has row-level security, a new index type, and performance enhancements for large servers. This talk covers the top ten new features that appeared in the Postgres 9.5 release. It will also cover some of the major focuses for post-9.5 releases.

  • Nikolai Ryzhikov
    Nikolai Ryzhikov Health Samurai
    45 мин
  • Valentine Gogichashvili
    Valentine Gogichashvili Zalando

    Data Integration in the World of Microservices

    Since its launch in 2008, Zalando has grown with tremendous speed. The road from startup to multinational corporation has been full of challenges, especially for Zalando's technology team. Distributed across Berlin, Helsinki, Dublin and Dortmund — and nearly 900 professionals strong — Zalando Technology still plans to expand by adding 1,000 more developers through the end of 2016. This rapid growth has showed us that we need to be very flexible about developing processes and organizational structures, so we can scale and experiment. In March 2015, our team adopted Radical Agility: a tech management strategy that emphasizes Autonomy, Purpose, and Mastery, with trust as the glue holding it all together. To make autonomy possible, teams can now choose their own technology stacks for the products they own. Microservices, speaking with each other using RESTful APIs, promise to minimize the costs of integration between autonomous teams. Isolated AWS accounts, run on top of our own open-source Platform as a Service (called STUPS.io), give each autonomous team enough hardware to experiment and introduce new features without breaking our entire system.

    One small issue with having microservices isolated in their individual AWS accounts: Our teams keep local data for themselves. In this environment, building an ETL process for data analyses, or integrating data from different services, becomes quite challenging. PostgreSQL's new logical replication features, however, now make it possible to stream all the data changes from the isolated databases to the data integration system so that it can collect this data, represent it in different forms, and prepare it for analysis.

    In this talk, I will discuss Zalando's open-source data collection prototype, which uses PostgreSQL's logical replication streaming capabilities to collect data from various PostgreSQL databases and recreate it for different formats and systems (Data Lake, Operational Data Store, KPI calculation systems, automatic process monitoring). The audience will come away with new ideas for how to use Postgres streaming replication in a microservices environment.