Linux IO internals for PostgreSQL administrators

Input-output (IO) performance issues have been on DBAs’ agenda since the beginning of databases. The volume of data grows rapidly and time is of an essence when one needs to get necessary data fast from the disk and, more importantly, to the disk.

For most databases it is relatively easy to find checklist of recommended Linux settings to maximize IO throughput and, in most cases, this checklist is indeed good enough. It is however essential always to understand how the optimisation of those settings actually works, especially, if you run into corner cases.

In the modern world, an increasing number of IT companies are moving their resources to the cloud and Zalando is not an exception. A rapid growth our company is experiencing along with an adoption of microservices were the main driving forces behind the changes introduced into the deployment procedure of new PostgreSQL clusters and the solution of the automatic failover problem. The majority of existing solutions for automatic failover require manual configuration of every cluster instance and complicates provisioning new clusters and new nodes into existing cluster.

Postgresql Functions and ORM cooperation at Django example

We will discuss some tricks and patterns while developing web-application architecture based on ORM technology with Postgresql functions implementation with Python Django as an example. Also we will consider some ways of building the business logic inside a database with keeping Django framework opportunities and implementation of it's built-in admin panel.

VIDEO

Dynamic Compilation of SQL Queries in PostgreSQL Using LLVM JIT

Currently, to execute SQL queries PostgreSQL uses interpreter, which implements Volcano-style iteration model. At the same time it’s possible to get significant speedup by dynamically JIT-compiling query “on-the-fly”. In this case it’s possible to generate code that is specialized for given SQL query, and perform compiler optimizations using the information about table structure and data types that is already known at run time. This approach is especially important for complex queries, which performance is CPU-bound.