PGConf.Russia 2025

Dozens of PostgreSQL instances have replaced a single large Oracle database. One of our key business systems was completely redesigned and rebuilt on an import-independent solution.  

This talk will cover the unique challenges, architecture, issues, and operational aspects of this migration. Expect an insightful deep dive into the process!

Where I used to work, we had pushed ElasticSearch to its breaking point. We needed an even more scalable replacement for a write-heavy, read-seldom workload. So we built one on PostgreSQL. Now, many of us are building the successor as an open source project. 

This talk goes over the design of Bagger (named after the giant mining machines), which can manage logs into tens or hundreds of petabytes. More than just a review of the architecture, this talk focuses on the whys and the tradeoffs made in the design. 

The talk is intended both to showcase how programmable and powerful PostgreSQL is, but also illustrate the fundamental tradeoffs which must be faced when pushing any technology into the big data space.

The visibility map in PostgreSQL is an important mechanism for optimizing database performance, accelerating read operations, data cleanup, and indexing. Despite some limitations, such as memory consumption and fragmentation, its use can significantly enhance performance in high-load systems. Effective utilization of the visibility map requires proper configuration and monitoring, which is especially crucial in systems with large data volumes and high transaction concurrency.

In this talk, I will challenge architectural stereotypes about the impossibility of communication with external systems directly from a DBMS. PostgreSQL offers vast communication capabilities; you just need to know which extensions to use.

And when extensions aren't enough, you can create your own. For the Luxms BI project, we needed communication with NATS, so we developed our own extension in Rust. I’ll share the GitHub link and my experience of developing PostgreSQL extensions in Rust during this talk.