Very large tables in PostgreSQL. Or how to make 60+ Tb to 10+ Tb

The talk will describe how we have implemented storage of large tables (+1 billion rows per day). The project exists in production 2 years. The total amount of data - 300 Tb (25 PostgreSQL servers * 2 Data Center). I'll tell about mistakes in organization of large tables storage in the initial phase of the project, and how these mistakes were corrected. I'll also talk about how to organize the data rotation and archiving. I voiced questions about what we were missing in PostgreSQL 9.4 out of what appeared in the 9.5 and 9.6. And also, what new features we are waiting for new releases of PostgreSQL.

RUM indexes and their applications

I want to present a new custom access method, which extends the current GIN capabilities using additional information stored in posting tree/list. For example, positional information as an additional information allows new AM returns results in relevance order, which could considerably improve execution time of full text queries.

Tools for server code debugging, profiling, and tracing

Debugging, profiling, and tracing of the executed commands play an important role in development of any applications. This is also true for developing stored procedures in DBMS.

PostgreSQL offers various tools (both built-in and external) for these purposes.

In this talk, we will provide an overview of the available tools and their advantages and disadvantages, as well as a detailed demo of their use cases.

This talk is a part of a basic course for backend application developers (DEV1), which “Postgres Professional” company plans to announce in the near future.

Connection pooling at any scale

It's not a secret that PostgreSQL connections are expensive so you should save them. To solve this problem there are PgPool-II and PgBouncer for quite a long time. At Yandex tens of thousands of connections to a single database is not a surprise so we use pgbouncer since time immemorial. This talk gives an overview of problems we faced and ways to solve them.

VIDEO