PgConf.Russia 2017

This tutorial shows how to properly monitor PostgreSQL. We will discuss the mamonsu utility, see how to configure it, examine its hidden features and learn how to extend them.

Experience we've got after 5 years of developing, deploying and improving BI system http://colibri365.ru used in government. I would talk about government IT reality and our way over it. Postgres performance improvements, using of latest features, overwriting of user generated queries to help query optimizer and other tweaks and hacks to tackle limited hardware problems. These lead us to number of computer science papers and (now committed) patches to Postgres (see Andrey Borodin talks for details).

VIDEO

I will tell you about why Postgres is first-choice product as a foundation for your BI system with classical OLAP workload. Briefly it will be said about existing open source BI solutions.

I will also describe specific of our architecture, why we chose snowflake scheme and how we are doing extract, transformation and load procedures. It will be mentioned about special Postgres tuning for OLAP and massive data bulkload workloads. Also I will let you know about Postgres usage as a column database with cstore_fdw by Citus and results achieved. Cons and problems of our approach will be described in the end of the talk.

VIDEO

Database sharding enables a distribution of the database over a large number of machines, greatly improving performance. With the advent of Foreign Data Wrappers (FDW), it's possible to consider a database sharding in PostgreSQL with acceptable level of code changes using FDW. We've been working on enhancing around FDW infrastructure such as foreign table inheritance and pushing down so that PostgreSQL can execute the distributed query efficiently using FDW. In this talk, I'll cover what FDW-based sharding is and what use-cases it can cover. And then I'll demonstrate how to build sharding and describe our achievement of a FDW-based sharding in PostgreSQL community. Finally, I'll describe further enhancements to FDW such as Async Execution and Distributed Transaction Support.