PGConf.Russia 2025

The focus of the presentation is on the challenges of building an application solution on PostgreSQL, specifically a high-load analytical data warehouse. Using the case of the company EVRAZ, we will demonstrate the impact of applying the Data Vault methodology on PostgreSQL and Greenplum databases for developing a unified production performance system.

We will discuss the difficulties and solutions, showcasing query plans for tasks such as updating directories and handling large objects, along with optimization examples. We will highlight coding nuances, problems related to populating the model, and issues with querying from the model.

This session will be useful for those planning to use or already facing challenges with the Data Vault methodology and performance issues in DWH on the open-source stack. We will compare technical implementation options for the Business Vault model layer, considering the specifics of PostgreSQL and Greenplum.

We will also cover 5 real problems that arise when operating a DWH and their solutions:

1. Transferring Business Vault object assembly logic from PostgreSQL to Greenplum.
2. Slower ETL performance when building the current state of Business Vault in PostgreSQL.
3. Slower Data Lineage construction in PostgreSQL and Greenplum.
4. Slow satellite queries in Greenplum.
5. Slow queries with "IN" or "OR" in the Business Vault layer.

I’ll talk about the challenges you’ll face if you decide to implement a new TableAM. What to choose: Generic XLog or Custom RMGR? Why use a Custom SMGR? How to integrate PostgreSQL allocators into third-party libraries, even if they don't officially support it? What’s missing for a columnar engine (including vectorization and late materialization), and how can we work around those limitations?

In the second part, I’ll dive into the internals of pgpro_tam — a new native table engine for OLAP that supports standard data formats, various SMGRs, and, if needed, third-party schedulers and execution engines, all while adhering to ACID principles. This is designed to achieve the fastest analytics on PostgreSQL (not just plugging in DuckDB).

Although we often don't like to talk about it, database administration carries with it a large amount of stress. This is particularly true when we have to take on late night support duties. In this talk you will learn: 1. The basics of the physiology of stress. This is not a physiology lecture, but as engineers we need enough to build a mental model. 2. What kinds of mistakes we make when under stress. 3. How to measure stress in your own life and recognize it in others. 4. What tools are helpful to manage short- and long-term stressors. Managing short-term stress is important in avoiding becoming overwhelmed when problems occur, but managing longer-term stressors is just as important in avoiding problems in the first place.

In this talk, we’ll:

• Share our experience of reworking the ECM system and platform that was originally built on C.Net + MS SQL for Windows to run on Linux + Postgres, while maintaining the option to install and operate on Windows + MS.

• Discuss migrating clients to new platforms with data migration.

• Share insights from operating the system with a high-load client (PG servers with 192 cores and 3 TB of RAM).