PGConf.Russia 2025

Many developers often face performance issues in the systems they develop. One common solution for optimizing slow business processes is parallelization. But what do you do if the bottleneck is the data insertion into the database, which needs to maintain atomicity?

In this talk, I’ll explain how to speed up data insertion by parallelizing the process in Spring, while ensuring the atomicity of the entire operation. We'll cover batch updates in Spring and PostgreSQL, discuss why updates are heavy operations, and explore ways to speed up the process in the current tech stack. Additionally, I will present other approaches to maintaining atomicity and demonstrate their differences in benchmarks.

This will be useful for practicing engineers.

The focus of the presentation is on the challenges of building an application solution on PostgreSQL, specifically a high-load analytical data warehouse. Using the case of the company EVRAZ, we will demonstrate the impact of applying the Data Vault methodology on PostgreSQL and Greenplum databases for developing a unified production performance system.

We will discuss the difficulties and solutions, showcasing query plans for tasks such as updating directories and handling large objects, along with optimization examples. We will highlight coding nuances, problems related to populating the model, and issues with querying from the model.

This session will be useful for those planning to use or already facing challenges with the Data Vault methodology and performance issues in DWH on the open-source stack. We will compare technical implementation options for the Business Vault model layer, considering the specifics of PostgreSQL and Greenplum.

We will also cover 5 real problems that arise when operating a DWH and their solutions:

1. Transferring Business Vault object assembly logic from PostgreSQL to Greenplum.
2. Slower ETL performance when building the current state of Business Vault in PostgreSQL.
3. Slower Data Lineage construction in PostgreSQL and Greenplum.
4. Slow satellite queries in Greenplum.
5. Slow queries with "IN" or "OR" in the Business Vault layer.

Experience Report addressing the manual migration of MS SQL Server Stored Procedures and Table-Valued Functions to PL/pgSQL. We chose a manual migration from T-SQL over using a PostgreSQL Extension with an automated translation. The motivation was that the T-SQL code contains non-trivial business logic for which we wanted idiomatic PL/pgSQL code. 

The T-SQL Code used features like 

• Mix of Stored Procedures and Table-valued Functions

• Table variables, (user-defined) table types 

• Recursive Common Table Expressions 

• Optional parameters 

The migration required

• Adopting naming conventions 

• Renaming parameters and local variables 

• Maping table types 

• Mapping table-valued parameters to arrays 

• Mapping table-valued return types to SETOF record

• Translating between arrays and tables 

In this session we will migrate a chain of dependend functions T-SQL functions to PL/pgSQL. We will work around quirks in both T-SQL functions and PL/pgSQL.