PGConf.Russia 2018

Lately, the PostgreSQL is more widely being used for Enterprise. Our company, Ingram Micro Cloud, is one of the first companies that did it. We have been using PostgreSQL for many years as the main DBMS for our products. In our report, we want to tell about the evolution of our High Availability (HA) PostgreSQL cluster. We will tell about how quickly we implemented the solution using pgpool-II, wrote failover scripts, tested Postgres-XL, and came up with unusual configurations of Stolon. We will also cover the problems of load balancing, pooling connections, and backups.

In the pitch I will talk about the most important nuances of deployment and operations of the distributed analytical open-source database based on PostgreSQL - Greenplum. I will analyze the typical mistakes in its use, give the best practices and warn about bottlenecks.

In this talk, I will tell you the story of how a bunch of sysadmins got sick of having to resuscitate their petabyte-sized Elasticsearch cluster and decided to replace it with some tried technologies: PostgreSQL, Kafka, a bit of Redis, lots of glue, and the typical sysadmin stubbornness. The result is Bagger: the sysadmin answer to Big Data. A fast, fairly reliable, fault-tolerant store, used mostly for logging timestamped events for some amount of time. Bagger is named the Bagger series of bucket-wheel excavators, feats of German engineering and some of the largest land vehicles ever produced by man. Just like the excavators that dig through tons of material, our Bagger digs through tons data.

Liquibase is a very convenient tool for sequential database migration, used both on our projects and in a large number of other projects and frameworks. It allows you to keep the code of the database together with the application code in VSC, track the attempts of repeated migrations and much, much more. But sooner or later the project grows, the data occupy terabytes, and liquibase still rolls the migration sequentially.

We could not afford to deploy migrations for 100 hours and came up with a framework for liquibase that expanded its capabilities and allowed to execute a whole series of scripts in parallel or to split one large migration into small partitions and migrate them in parallel.