PgConf.Russia 2017

Nowadays one can make a decent urban research based simply on public datasets, making interesting and unexpected insights. In the presentation, I'll show examples of these calculations in PostGIS, the industry standard de-facto.

But just PostGIS is not enough. You need tools to import, verify and visualize the data. It's critically important to visualize the data live, to debug your calculations and shorten iterations. I'll describe all these steps:

1. Collecting the data: public API, OpenStreetMap; direct user input.
2. 3rd party APIs for calculations.
3. Visualization of GIS and other sorts of data: QGIS, Matplotlib, Zeppelin integrated with PostGIS.
4. Debugging the calculations: live visualization (Arc, QGIS, NextGIS Web)
5. Scripting and minimizing the chores: Makefile, Gulp

Currently, to execute SQL queries PostgreSQL uses interpreter, which implements Volcano-style iteration model. At the same time it’s possible to get significant speedup by dynamically JIT-compiling query “on-the-fly”. In this case it’s possible to generate code that is specialized for given SQL query, and perform compiler optimizations using the information about table structure and data types that is already known at run time. This approach is especially important for complex queries, which performance is CPU-bound.

Streaming replication has been introduced in 2010 and quickly became one of the most popular features of PostgreSQL. Today, it is hard to imagine PostgreSQL installation without streaming replication. With its stability, high efficiency and ease of configuration one would have thought it is an optimal feature. However, while using it you might sometimes enter murky waters. This often can be resolved by using a combination of built-in and third party troubleshooting tools. In my talk I will provide an overview of these tools and explain how with their help one can diagnose, understand and eliminate problems related to streaming replication. I will also go through the most frequent issues occurring when streaming replication is used and will propose possible solutions. This talk is primarily aimed at DBAs and system administrators who use PostgreSQL in their day-to-day.

VIDEO

It often happens that you already have PosgtreSQL in production, but you don’t have a DBA. To demystify the PostgreSQL database, I’ll tell you how to troubleshoot various problems while working with PosgtreSQL. We will try to understand how to answer "routine" questions of a typical system administrator: - Is everything OK with the database? - What consumes DB server resources? - What to optimize first to reduce resource consumption?

VIDEO