Talks

How to migrate a high-loaded information system based on 1С:Enterprise from MS Windows/MS SQL Server to Linux/PostgreSQL and don't face considerable pains?

We'll share our migration strategies and what was key to its success. We'll talk about projects involving migrations and pay attention to some nuances.

Slides

I will share the experience of migration project of an information system from a Microsoft SQL Server DBMS to Postgres PRO. The owner of the DBMS is Rosagroleasing JSC.

The volume of the DBMS database is about 6 TB (more than 300 tables, up to 4 billion records in each). The team adapted 15 ETL packages and 131 procedures with full functionality and also partial refactoring of the source code. Systems' performance has remained the same.

Slides

Since PostgreSQL 9.5 logical replication has replication origin infrastructure for replication progress tracking. Now 16th version is coming with feature which made bi-directional replication setups possible using said infrastructure. In earliest reviews it got a nickname of "native multimaster" for cluster, interconnected with said replication, bearing some resemblance to PostgresPro's Multimaster. We're going to figure out degree of it and explain insides of them. Talk expects you to have a basic knowledge of WAL insides and it's role in PostgreSQL functioning.

Slides

Three years ago we successfully migrated our application from Oracle DBMS to vanilla PostgreSQL. After that we faced with "childhood" diseases of our application on the new DBMS, which were successfully cured, and some "chronic diseases", which took much longer to deal with. One of the most memorable problems was the problem of DB performance degradation. The source of this problems was insufficient vacuuming of our database. The experience of understanding and solving this problem is offered to your attention in the form of practical recommendations to prevent DB tables and indices bloat and setting up PostgreSQL VACUUM/autovacuum parameters.

Slides

I will talk about the common mistakes when setting up a Corosync-Pacemaker failover cluster. Often these mistakes lead to fatal consequences, and, as a result, to the rejection of the chosen solution in favor of others. Would you like to get a recipe for the "right" cluster?

Slides

The history of import substitution did not start last year after well-known events. Its launch dates back to 2014. It was from this year that state and near-state companies began to think of switching to the so called "recommended software". One of these companies approached us back in 2020 with a project to move from Oracle to PostgreSQL. This project was designed to solve the accumulated architectural problems (imperfect storage system for telemetry data, the DBMS itself worked inside a virtual machine), and optimize the use of disk space (make space in the main storage, debug saving archived data, ensure correct backup). Since the customer's system should have worked uninterrupted 24/7, it was necessary to switch from one DBMS to another "seamlessly" without downtime, with simultaneous operation of both to ensure step-by-step translation of subsystems and the ability to control the correctness of the data. And, of course, the work had to be completed as quickly as possible.

In the report we will discuss how we managed to solve this case.

Slides

As we have large 1C+PostgreSQL database setups, questions often arise, the answers to which are not so easy to find even in the documentation. I would like to share my experience in solving such issues gained during several migrations of 1C from MS SQL to PostgreSQL made for clients from the RBC500 rating. This talk will cover the following points: How in-depth your statistics should be and why it can be dangerous? How can the creation of an explicit and implicit temporary table provoke the DBMS server downtime and how can we avoid it? In what case will the DBMS process be killed by the operating system due to overuse of RAM and what to do about it? Why having one database per cluster is good and why having multiple databases on one DBMS cluster is bad? If we are stick to the "1 cluster = 1 database" approach, what about server resources for test and development environments? Backups in PostgreSQL: what backup options we have

Slides

How to thoughtfully recover corrupt data...

Slides

My presentation "Explaining the Postgres Query Optimizer" covers the details of query optimization, optimizer statistics, joins, and indexes. This talk covers 40 other operations the optimizer can choose to handle complex queries, large data sets, and to enhance performance. These include merge append, gather, memoize, and hash aggregate. It explains their purpose and shows queries that can generate these operations.

This is a new talk; draft slides are at https://momjian.us/main/writings/pgsql/beyond.pdf

Slides

Every DBA has in one way or another experienced the situation when PostgreSQL tables and indexes grow significantly in size. While looking for the reason for such behavior, we often conclude that database objects have "bloated". In this talk, we'll discuss the reasons behind bloating, create a testing environment to define the best method to reduce bloating. We'll also compare several anti-bloating utilities and get familiar with one more tool that helps us to efficiently struggle against bloating. We expect this presentation to become helpful for PostgreSQL DBAs of any experience level.

Slides

Delta migration case using Debezium is considered in details as well as the process of change data capture, sinking, applying configurations and load testing.

Slides

In this talk I’ll show some new abilities in postgres observability. There will be Graphana visualization of pg_profile/pgpro_pwr data, manual highlighting of interesting objects in pg_profile/pgpro_pwr reports, further development of extended vacuum statistics in PostgresPro releases and a new session and query tracing technique in pgpro_stats extension

Slides

One of the main needs of high-load database projects is a right configured load balancer within the database cluster. Most of the solutions I worked with, including Pgpool-II, could not fully suit business requirements. In this article me as a network engineer with broad experience will cover tricky configurations of the Pgpool-II operating in the “Master-Master” mode, as well as consider cases in which similar solutions are indispensable.

Slides

I'll show in PostGIS how everyone can analyze the geodata of the entire Earth and get answers to their global questions in minutes and seconds.

When you use a taxi in small towns, calling a car by phone, then with a high probability your trip is charged by the program based on OSM data. For billing, one of the routing packages is used. Through this use case, the taxi company employees put the house number and street on the buildings and contribute not only to their business, but also to OpenStreetMap.

The data analytics scenario also includes the tasks of where it is better to place an outlet so that buyers come to it. Again, data on walking distance and population of the surroundings can be extracted from geodata. You can calculate the value of real estate based on many factors related to the location of the object and its surroundings.

Scientists can build predictive models to predict epidemics, urban evolution, plan recreational areas and development of existing territories based on open geodata.

Well, you can answer any geography question that comes to your mind: calculate the area of cities and buildings, the length of roads and extract the names of cities, regions and islands. You can, for example, become a champion in the game of "Cities" or establish a new service for renting electric scooters. Everything is limited only by your imagination.

I published https://github.com/igor-suhorukov/openstreetmap_h3 - my project of high performance data loader, which allows you to perform geoanalytics on data from OpenStreetMap in PostGIS. It transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions schema. Columnar storage option activate CitusDB extension in PostgreSQL to speedup aggregation queries.

Slides

How does network latency affect performance when using synchronous replication? If a primary node fails, how long will our cluster not be reachable? How much data can be lost? Will there be any anomalies when reading from standby? We are designing our network architecture, give us a minimum RTT value for cluster solution! Many DBAs have to face these issues when designing and implementing cluster solutions. In this short talk we will tell how to quickly estimate RTT, RTO, RPO values, with the help of which Open Source tools one can test cluster solution and experimentally verify the design parameters, which metrics values can be used as a guide in design.

Slides

The major task of workload monitoring tools is to provide measurable statistics about most resource intensive activities in the DBMS. They can't solve any issues but the points for concentrating optimization efforts can be highlighted. We’ll talk about the extensions pg_profile and pgpro_pwr, their architecture, features and use cases.

Slides

Slides

A report on our experience of implementing PostgreSQL instead of Oralce in a highly loaded system with a microservice architecture that processes several terabytes of data. Let's talk about how we went from piloting Greenplum to moving to several interacting PosgreSQL databases with different load profiles, and about the problems we encountered in the process.

Slides

The following topics will be covered as part of the presentation:

• Architecture of logical replication
• Publisher introduction
• Subscriber introduction
• Data syncronization introduction
• Logical decoding
• Replication slot
• output plugin

Slides

This report is primarily intended for companies, specifically their IT departments, that are running Russian systems 1C 8.x and have the ability to work with both MSSQL Server and PostgreSQL. We live in a unique time when having a heterogeneous IT system (a system with a distributed architecture where each database instance runs under different databases and/or has a different data structure) is justified both economically and in terms of potential risks. On the one hand, we store data and use databases with predictable behavior and open source code, regardless of the political situation. On the other hand, with this approach, we take advantage of all the benefits (primarily performance) of a powerful database supported by the largest vendor, albeit a hostile state to us. Now is the time to assess risks with the necessary level of paranoia. After all, data can be corrupted not only at the level of storage logic, but also by the method of additional "hidden" calls of Delete/Update constructs at the engine level of the database. Therefore, it is now relevant not only to monitor the performance of the IT system, but also to audit data and their timely integrity. It is necessary to implement data verification procedures, it is necessary to implement procedures for closing periods, procedures for fault-tolerant storage. And accordingly, to provide for various threat models in data recovery procedures. The report presents options for countering such threats and scenarios for the most seamless transition of large databases to PostgreSQL, since it is precisely for such databases that the problem of translating to a new DBMS is particularly acute.

Slides

We offer to your attention our experience in data migration and the Ora2PgCopy program written in Java for high-speed data transfer from Oracle to Postgres, which is used after creating tables and transferring the program code of application systems. High data transfer speed is provided by using the Postgres command “copy”, using multithreaded Java technology for file processing, managing the nologged/logged table option, and supporting LOB and CLOB data types. According to the test results, Ora2PgCopy works noticeably faster than such analogues as: Ispirer (convertum), oracle_fdw, ora2pg, Pentaho kettle. Ora2PgCopy can function as a module as part of the LUI4ORA2PG migration automation system or independently of it. The history of the growth of the Live Universal Interface (LUI) web application development tool and the LUI4ORA2PG migration tool can be found in previous presentations at PGConf conferences: https://pgconf.ru/2019/118109 , https://pgconf.ru/201911/264095 , https://pgconf.ru/2020/262456, https://pgconf.ru/2021/288310, https://pgconf.ru/2022/316022.

Slides

Of the relational databases, PostgreSQL is fairly unique in the indexing capabilities it offers. While most of us are familiar with the use of indexes to speed performance when filtering on columns, PostgreSQL indexes can do far more than this.

This talk will focus on cases where difficult problems were solved through the creative use of indexes. Each of these cases is from an episode in my career.

In this presentation, you will learn:

• The general index access methods PostgreSQL supports out of the box
• How the planner uses indexes (only on a high level)
• Functional indexes
• Creative use of Index Only scans.

This is a talk for all audiences. The talk is primarily for beginner to intermediate users, but should have enough information for more advanced users to get some insight or inspiration from the talk as well.

Slides