PGConf.Russia 2025

The focus of the presentation is on the challenges of building an application solution on PostgreSQL, specifically a high-load analytical data warehouse. Using the case of the company EVRAZ, we will demonstrate the impact of applying the Data Vault methodology on PostgreSQL and Greenplum databases for developing a unified production performance system.

We will discuss the difficulties and solutions, showcasing query plans for tasks such as updating directories and handling large objects, along with optimization examples. We will highlight coding nuances, problems related to populating the model, and issues with querying from the model.

This session will be useful for those planning to use or already facing challenges with the Data Vault methodology and performance issues in DWH on the open-source stack. We will compare technical implementation options for the Business Vault model layer, considering the specifics of PostgreSQL and Greenplum.

We will also cover 5 real problems that arise when operating a DWH and their solutions:

1. Transferring Business Vault object assembly logic from PostgreSQL to Greenplum.
2. Slower ETL performance when building the current state of Business Vault in PostgreSQL.
3. Slower Data Lineage construction in PostgreSQL and Greenplum.
4. Slow satellite queries in Greenplum.
5. Slow queries with "IN" or "OR" in the Business Vault layer.

This talk explores the challenges we encountered — and solved — while migrating massive databases from Oracle to PostgreSQL.

One of the most complex aspects of this process was rebuilding foreign keys (FKs). To overcome these challenges, we had to dive deep into the internal workings of FK creation and validation.

Key topics include:

Does ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY have an execution plan?

Can an ordinary user influence this process?

What locks are applied during FK creation and validation?

Also, we’ll introduce a first-principles method for investigating performance issues, applying it to a real-world FK creation bottleneck.

This talk addresses a common issue and touches a bit on application architecture. Temporary tables in database applications are typically used for several purposes. 

Firstly, they are used to store intermediate results when implementing complex data processing algorithms. Secondly, in the case of stored procedures, application servers often place large datasets into temporary tables when they are too large or inconvenient to pass as parameters to stored procedures. 

The handling of temporary tables in different DBMSs is implemented in various ways, which often complicates migration from these systems to Postgres. 

The drawbacks of temporary tables are well-known, which leads to a reasonable desire to replace them, where possible, with other methods that can achieve the same goal. This talk focuses on alternative mechanisms provided by Postgres to solve this problem.

In some cases, business logic is implemented directly within the database. It's well known that maintaining and developing business logic becomes much easier when it's covered by tests. However, backend tests often treat the database as an external resource and don’t always interact with a live database.

This talk will introduce an enhanced version of the pgTap library, designed not only to facilitate unit testing and CI for database code but also to accelerate the development process itself.