PGConf.Russia 2022

SPQR is a lightweight OLTP sharding solution written in Go. In this talk, I'm going to discuss design decisions that led us through FDW-based sharding, CustomNode-based sharding, C-implemented lightweight query routing, and finally to SPQR design.

In June 2020, AlfaStrakhovaniye was obliged to use RSA 2.0 government services along with other insurance companies. It was the only way to sell car insurance in Russia. AlfaStrakhovaniye is a leader in car insurance, and its applications degraded under the pressure of calls. It wasn't planned. Postgres used a huge amount of CPU and memory. Nobody could tell what was going on. Degradation started and stopped for no reason. Dev Team and Postgres Pro experts solved that problem. MyBatis in Java services was the killer. Our story is about the development of applications for heavy workloads and technical ignorance while using frameworks.

Logical decoding and replication slots introduced in PostgreSQL 9.4 (released in 2014) created a solid foundation for implementing built-in core logical replication in version 10 (released in 2017). Unfortunately, there are a few limitations that make logical replication not very useful in real-world scenarios. Logical decoding currently isn’t supported on the standby server, and PostgreSQL allows creating logical replication slots only on the primary server. Or in other words, logical slots are lost on failover/switchover.

Postgres hackers made many attempts to address the problem, and most of them resulted in not too much success. Although, there is one little function introduced in PostgreSQL 11 that made it possible to implement failover of logical replication slots externally.

In my talk I will tell a story of how Patroni solves the problem of logical replication slots failover without using invasive third-party extensions, dig down into some of the Postgres internals in order to prove why this approach is safe, and finally, we will discuss limitations and potential downsides of this solution.