Доклады

PostgreSQL 16: at the finish line

April 8, 2023 is the expected feature freeze date for PostgreSQL 16.

Some of the new features are already known, particularly, we can expect interesting changes related to security and logical replication. However, announcing all interesting patches before the end of the March CommitFest is impossible. Based on the experience of previous releases, we can assume that the most interesting patches will be committed during late March and early April..

That's why the contents of this talk will remain unknown till the very last moment, the author of this talk will also discover it in due time.

Review of some historical CVEs

I'm not a real security expert. But this will help me to do ELI5 explanation of the funniest historical Postgres CVEs.

Experience in porting the production management system database from Oracle DBMS to PostgresPro DBMS in a manufacturing enterprise

The practice of transferring structure, logic and data from Oracle DBMS to PostgresPro DBMS. Features and main difficulties of migration. Advantages of PostgresPro in terms of porting logic.

ORM LinqToDb: how to make the most out of Postgres

ORM libraries are widely used in backend development – let's take this for granted. ORM libraries are improving, developers are learning from their own and others' mistakes, and many have learned to write code for ORM that generates quite optimal queries for typical operations. Yet, limited functionality of existing tools is another problem related to ORM. This is especially true for Postgres, which has a lot of operators and constructs (both in the core and in extensions) that go beyond the SQL standard.

With LinqToDb ORM library you can write extensions that implement any SQL operator or construct very easy, literally using just one line. Full-text search, KNN, CTE (including recursive ones), working with arrays and jsonb, array_agg and jsonb_object_agg, table functions and LATERAL JOIN - all this can be easily used in ORM, which I will demonstrate during this talk!

Experience of migration a highly loaded system from DB2 for z/OS to PostgresPro in multimaster configuration

The developer shares his experience of migration a highly loaded system in the field of railway transport from IBM DB2 for z/OS to PostgresPro in a multimaster configuration. The main focus is on the nuances of working with a cluster operating in multimaster mode, what problems had to be faced and how to solve them.

PostgreSQL vs Redis: Making the Right Choice

With the rise of NoSQL databases, a number of falsehoods have flourished regarding how to choose a database engine. This talk focuses specifically on Redis and PostgreSQL, and why one might choose one or the other.

At small scales, we can often get by thinking of database servers as black boxes, but as we scale, the internals and architecture become more and more important. This talk focuses on behavior of these systems at scale and under load.

In this presentation you will learn:

• How Redis and PostgreSQL differ architecturally
• How differences in architecture affect scalability and performance
• Cases here Redis is the clear winner
• Cases where PostgreSQL is the clear winner

Additionally, some notes will be offered in terms of where PostgreSQL can improve in to compete with the sorts of workloads that generally favor Redis.

A set of Ansible roles for PostgreSQL

There is set of Ansible roles for any Postgres configuration: a standalone instance; a streaming replica; patroni+etcd; stolon+etcd; patroni+consul; stolon+consul. In this talk, we will discuss why using this set of Ansible roles is better than other solutions.

How we switched Oracle for PostgreSQL for a client, before it became mainstream

The history of import substitution did not start last year after well-known events. Its launch dates back to 2014. It was from this year that state and near-state companies began to think of switching to the so called "recommended software". One of these companies approached us back in 2020 with a project to move from Oracle to PostgreSQL. This project was designed to solve the accumulated architectural problems (imperfect storage system for telemetry data, the DBMS itself worked inside a virtual machine), and optimize the use of disk space (make space in the main storage, debug saving archived data, ensure correct backup). Since the customer's system should have worked uninterrupted 24/7, it was necessary to switch from one DBMS to another "seamlessly" without downtime, with simultaneous operation of both to ensure step-by-step translation of subsystems and the ability to control the correctness of the data. And, of course, the work had to be completed as quickly as possible.

In the report we will discuss how we managed to solve this case.

Oracle-like packages in Postgres Pro Enterprise 15

One of the main problems when migrating from Oracle to PostgreSQL is PL/SQL packages.

The PostgreSQL DBMS lacks such functionality - this leads to the need to rewrite (up to its redesign!) the Oracle PL/SQL code to PL/pgSQL.

With the new version of its flagship Postgres Pro 15 Enterpise distribution, Postgres Pro has added support for package functionality: the PL/pgSQL syntax has been extended and new mechanisms have been added to the stored procedure runtime environment. The report will focus on supporting the functionality of packages in Postgres Pro 15 Enterpise: - grouping types of variable procedures and functions into a package; - support for the package initialization section; - support for global package variables available throughout the "life" of the session.

This presentation will also discuss the features of migration of packages from Oracle to Postgres Pro Enterprise DBMS.

Tips and tricks for PostgreSQL operations for 1C

As we have large 1C+PostgreSQL database setups, questions often arise, the answers to which are not so easy to find even in the documentation. I would like to share my experience in solving such issues gained during several migrations of 1C from MS SQL to PostgreSQL made for clients from the RBC500 rating. This talk will cover the following points: How in-depth your statistics should be and why it can be dangerous? How can the creation of an explicit and implicit temporary table provoke the DBMS server downtime and how can we avoid it? In what case will the DBMS process be killed by the operating system due to overuse of RAM and what to do about it? Why having one database per cluster is good and why having multiple databases on one DBMS cluster is bad? If we are stick to the "1 cluster = 1 database" approach, what about server resources for test and development environments? Backups in PostgreSQL: what backup options we have

Pgpool-II in the "Master-Master" mode or Network engineer’s overview of the PostgreSQL load balances solution.

One of the main needs of high-load database projects is a right configured load balancer within the database cluster. Most of the solutions I worked with, including Pgpool-II, could not fully suit business requirements. In this article me as a network engineer with broad experience will cover tricky configurations of the Pgpool-II operating in the “Master-Master” mode, as well as consider cases in which similar solutions are indispensable.

Migration of high-loaded 1C solutions to Linux/Postgres infrastructure

How to migrate a high-loaded information system based on 1С:Enterprise from MS Windows/MS SQL Server to Linux/PostgreSQL and don't face considerable pains?

We'll share our migration strategies and what was key to its success. We'll talk about projects involving migrations and pay attention to some nuances.

How to put the whole world into a regular laptop: PostgreSQL and OpenStreetMap

I'll show in PostGIS how everyone can analyze the geodata of the entire Earth and get answers to their global questions in minutes and seconds.

When you use a taxi in small towns, calling a car by phone, then with a high probability your trip is charged by the program based on OSM data. For billing, one of the routing packages is used. Through this use case, the taxi company employees put the house number and street on the buildings and contribute not only to their business, but also to OpenStreetMap.

The data analytics scenario also includes the tasks of where it is better to place an outlet so that buyers come to it. Again, data on walking distance and population of the surroundings can be extracted from geodata. You can calculate the value of real estate based on many factors related to the location of the object and its surroundings.

Scientists can build predictive models to predict epidemics, urban evolution, plan recreational areas and development of existing territories based on open geodata.

Well, you can answer any geography question that comes to your mind: calculate the area of cities and buildings, the length of roads and extract the names of cities, regions and islands. You can, for example, become a champion in the game of "Cities" or establish a new service for renting electric scooters. Everything is limited only by your imagination.

I published https://github.com/igor-suhorukov/openstreetmap_h3 - my project of high performance data loader, which allows you to perform geoanalytics on data from OpenStreetMap in PostGIS. It transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions schema. Columnar storage option activate CitusDB extension in PostgreSQL to speedup aggregation queries.

The story on the development of a database change management solution and what we lacked in Liquibase and Flyway

We will tell you how we store database objects in the version control system. Then we compare the traditional approaches to manage database migrations and the one we preferred. We will talk about both methods advantages and disadvantages. Finally, we will present our change management solution - pgmig.

Struggle against database bloating

Every DBA has in one way or another experienced the situation when PostgreSQL tables and indexes grow significantly in size. While looking for the reason for such behavior, we often conclude that database objects have "bloated". In this talk, we'll discuss the reasons behind bloating, create a testing environment to define the best method to reduce bloating. We'll also compare several anti-bloating utilities and get familiar with one more tool that helps us to efficiently struggle against bloating. We expect this presentation to become helpful for PostgreSQL DBAs of any experience level.

Five shades of sharding

The importance of sharding is now colossal. The size of today's databases exceeds 100 terabytes, scaling up vertically, and adding replicas containing a complete physical copy of the database is becoming difficult, especially when computational resources are scarce. Database sharding is a way to scale horizontally by partitioning data between compute nodes that are independent of each other.

In the PostgreSQL world, there are well-known scaling tools - CitusDB, and Greenplum - as well as new generation solutions - Cockroach DB, Yugabyte DB, SPQR, and Shardman.

In this talk, we will discuss the difference between these implementations, the advantages, and disadvantages of these solutions, examine the current state of implementation of sharding in vanilla PostgreSQL, and also touch on another important topic - providing data integrity and consistency guarantees across a distributed cluster.

Workload monitoring extensions pg_profile/ pgpro_pwr. Architecture, setup, features and use cases

The major task of workload monitoring tools is to provide measurable statistics about most resource intensive activities in the DBMS. They can't solve any issues but the points for concentrating optimization efforts can be highlighted. We’ll talk about the extensions pg_profile and pgpro_pwr, their architecture, features and use cases.

New UUID versions

Primary key generation is a known issue quite efficiently though not ideally solved by sequences. There are still problems with distributed generation, real uniqueness and predictability. Initially, UUID was not intended for use as a DBMS key, but its new versions (6, 7 and 8) make it quite convenient to use them as both synthetic and natural keys.

Database upgrade with pg_copydb

pg_copydb is a new tool that automates database copy between two running Postgres servers. In this talk we will review the utility, share results of testing and practical use of utility for Postgres database upgrade.

Building the protection system based on Postgres DBMS

Why would you need the domestic DBMS? 1. Because of law requirements: only domestic DBMSes are appropriate for data protection. 2. In terms of formalization: currently, it is impossible to purchase MS in Russia legally. 3. In terms of pricing: the licensing mechanism of foreign DBMS systems significantly increases the expenditures on protection. 4. In terms of performance: it depends on the optimization of information protection tools, which are currently developed for domestic DBMS systems (in order to reveal why you may refer to the first paragraph). 5. In terms of practice: by the end of last year we’ve finished transition to the domestic databases and everything works smoothly. What else does an information security officer need?

Vacuum Therapy: Treatment of Chronic DB Disease

Three years ago we successfully migrated our application from Oracle DBMS to vanilla PostgreSQL. After that we faced with "childhood" diseases of our application on the new DBMS, which were successfully cured, and some "chronic diseases", which took much longer to deal with. One of the most memorable problems was the problem of DB performance degradation. The source of this problems was insufficient vacuuming of our database. The experience of understanding and solving this problem is offered to your attention in the form of practical recommendations to prevent DB tables and indices bloat and setting up PostgreSQL VACUUM/autovacuum parameters.

Let's get acquainted with GEQO in 20 minutes

-----------------------------------------------------------QUERY PLAN--------------------------------------------------------------
Hash Join
  Hash Cond: (Subject = GEQO)
  -> Hash Join
        Hash Cond: (**Optimizer task = choose the best query execution plan**)
        -> Seq Scan on **The number of potential plans grows exponentially as the number of tables in a query increases**
        -> Hash
              -> Seq Scan on **PostgreSQL solves this problem by using the genetic optimizer (GEQO)**
  -> Hash
        -> Seq Scan on **Topics of the report:**
              Filter: (**(What is GEQO)** AND **(Pros and cons)** AND **(How it works)**)
(10 rows)