PGConf.Russia 2018

In the last six months I have been working with a massive OLAP environment with 20TB shards, spanning around 400TB of data. Come to listen to how we make it all work, the challenges, and the skills involved. This talk has very little in common with the 10TB and Beyond talk because the data environments are very different.

We will cover analytics performance, data alignment, reasons for building extensions in C, and moving data around between servers in multiple data centers.

This report is about my PostgreSQL extension pg_profile. This extension all you need to create periodic statistics snapshots and to keep them. You can build a report on one or many serial snapshots. This report will contain statistics information about database workload in specified time period. It is very useful to start investigation on performance degragation or excessive resoure consumption in the past.

One of advantages of document-oriented databases like MongoDB or Couchbase over RDBMSs is an ability to change the data scheme easily, fast and often. The traditional approach in RDBMS world involves doing an expensive ALTER TABLE operation, slow upgrade of an existing data, and stuff like this. This approach is often slow and inconvenient for application developers.

To solve this issue PostgreSQL provides JSON and JSONB datatypes. Also there are extensions like zson and pg_protobuf. From this talk you will learn how to work with these datatypes and extensions, their pros and cons and also related future work in this area.