PgConf.Russia 2016

Postgresql's buffer manager has parts where it's showing its age. We'll discuss how it currently works, what problems there are, and what attempts are in progress to rectify its weaknesses.

• Lookups in the buffer cache are expensive
• The buffer mapping table is organized as a hash table, which makes efficient implementations of prefetching, write coalescing, dropping of cache contents hard
• Relation extension scales badly
• Cache replacement is inefficient
• Cache replacement replaces the wrong buffers

Multicorn is a generic Foreign Data Wrapper which goal is to simplify development of FDWs by writing them in Python.

We will see:

• what is an FDW what Multicorn is trying to solve how to use it, with a brief tour of the FDWs shipping with Multicorn.
• how to write your own FDW in python, including the new 9.5 IMPORT FOREIGN SCHEMA api.
• the internals: what Multicorn is doing for you behind the scenes, and what it doesn't

After a presentation of FDWs in general, and what the Multicorn extension really is, we will take a look at some of the FDWs bundled with Multicorn.

Then, a complete tour of the Multicorn API will teach you how to write a FDW in python, including the following features:

• using the table definition
• WHERE clauses push-down
• output columns restrictions
• influencing the planner
• writing to a foreign table
• IMPORT FOREIGN SCHEMA
• ORDER BY clauses pushdown
• transaction management

This will be a hands-on explanation, with code snippets allowing you to build your own FDW in python from scratch.

Currently, PostgreSQL uses the interpreter to execute SQL-queries. This yields an overhead caused by indirect calls to handler functions and runtime checks, which could be avoided if the query were compiled into the native code "on-the-fly" (i.e. JIT-compiled): at a run time the specific table structure is known as well as data types used in the query. This is especially important for complex queries, which performance is CPU-bound. At the moment there are two major projects that implement JIT-compilation in PostgreSQL: a commercial database Vitesse DB and an open-source project PGStorm. The former uses LLVM JIT to achieve up to 8x speedup on selected TPC-H benchmarks, while the latter JIT-compiles the query using CUDA and executes it on GPU, which allows to speed up execution of specific query types by an order.

Our work is dedicated to adding support for SQL query JIT-compilation to PostgreSQL using LLVM compiler infrastructure. In the presentation we'll discuss how JIT-compilation can be used to speed up various stages of query execution in PostgreSQL, and the specifics of translating an SQL query into LLVM bitcode to achieve good performing native code. Also we'll present preliminary results for our JIT-compiler on TPC-H benchmark.

My experience of working with PostgreSQL has provided clear understanding of its main advantages, making us choose and recommend choosing it.
1. Beginning
2. Documentation
3. Community
4.1 Transactional DDL
4.2 WAL and True Physical Replication
4.3 Transactional Snapshot and True Logical Replication and PGQ
4.4 Exciting extensibility
5. Success