Доклады

Migration automation tools: practice of creating and using

A story about practical approaches and tools for automating migration using 2 examples: a database with a structure change in the new version and a database with a significant content of the application's business logic.

It's working! What else do you need?

I have been working with 1C on PostgreSQL for 6 years, and the DBMS has changed significantly since I started. In this presentation, I'm going to cover the progress it has made. The users will, just as always, claim that this is not enough. So in addition to milestones and achievements, I'll tackle the possible feature requests from the maintenance viewpoint and explain some workarounds.

Development direction for Postgres Pro DBMS

Postgres Pro Enterprise is based on open source PostgreSQL. However, the difference is significant: the current version of Postgres Pro Enterprise has 40 more features that are not a part of PostgreSQL. 20 more mechanisms are being developed either as built-in features or separate products. This includes BiHA, DBaaS, pg_probackup, etc. In this presentation, we'll briefly discuss some of them.

ML in SQL

In modern data analysis, machine learning models are used as often as databases. Such IT giants as Google and Amazon have already combined them. Microsoft and Yandex are not far behind. Isn't it time to implement a machine learning model in PostgreSQL? In the report you will hear about the basics of machine learning, its implementation in databases and an example of realization as postres extension.

pgCodeKeeper - a tool to simplify work with PostgreSQL

We'll talk about the tool we created, which monitors changes in the database and creates scripts for migrations of selected objects both interactively and automatically.

Cluster software hardening

Hardening is the process of strengthening the security of a system in order to reduce risks from possible threats. In my presentation, I will tell you how to protect service cluster communications using TLS connections, in order to avoid accidental or unauthorized access to Patroni's REST API and ETCD storage.

Fencing in the cloud and more

One of the requirements for the operation of the Corosync/Pacemaker failover cluster is the fencing of the failed node. In virtual environments, fencing is implemented by disabling the virtual machine through a hypervisor, in a cluster on physical servers - through IPMI/ILO. But what if it is impossible to organize fencing, for example, in the cloud environment? During this presentation, I will list and explain alternative methods of fencing a failed node.

Oracle Apex/Forms on Postgres up to 100 times faster

Migration from Oracle DB/Application. What is the price?

Lowering the threshold for a programmer to enter the development of business applications using the Database-Centric approach on the Low Code Development Platform based on PostgreSQL.

Migrating large databases from Oracle to PostgreSQL using oracle_fdw

Recently, one of the most common migration methods has become migration using the free ora2pg tool. However, many note the extremely low performance of this tool, especially when it comes to databases of 1 Terabyte and above. This presentation describes how to significantly speed up database migration using oracle_fdw alone or in combination with ora2pg.

Postgres Pro Enterprise Manager - Database Control Panel

In this brief overview presentation we will discuss Postgres Pro Enterprise Manager (PPEM) capabilities, and how it helps DBAs to be more productive.

How to simplify the migration from Oracle to Postgres Pro

Postgres Pro is investing heavily to make it easy for customers to migrate to their database from Oracle Database.

This talk describes in detail the advanced Postgres Pro DBMS technologies designed to solve this problem:

• support for package functionality in PL/pgSQL, including the package initialization section and global package variables;
• ora2pgrpo utility for automatic conversion of package code from Oracle PL/SQL syntax to PL/pgSQL;
• PG Pro Application Info extension designed for instrumentation and monitoring of the state of sessions, including the execution of long operations (session longops) - a functional analogue of the package DBMS_APPLICATION_INFO in Oracle DBMS;
• new system packages UTL_MAIL and UTL_SMTP for sending emails from stored procedures to Postgres Pro DBMS (full functional analogue of the corresponding packages in Oracle DBMS);
• new system package UTL_HTTP for interacting with external sources from stored procedures in Postgres Pro DBMS (analogous to the package of the same name in Oracle DBMS);

Also in this talk there will be a short story about the directions of development of the Postgres Pro DBMS aimed at further simplifying migration from Oracle DBMS.

The hitchhikers' guide to the databases

There are a lot of different databases. We need some formal criteria to compare databases to each other.
The very first idea is to divide SQL and NoSQL.
NoSQL is a popular class of platforms developed in 2000s. Indeed, the rejection of SQL is not a fresh idea because there were predecessors of the relational database model, such as network and hierarchical models.
The fresh «NoSQL» stream consists of the graph, object, and key-value models.
Time-series, wide column, and «document-oriented» models are just extensions of the key-value model. Their advantage is the possibility to parse either key or value on a database server.
The facilities of SQL are much more extensive than the key-value interface. So, the simplified interface is just a charge for the ability to build a distributed database.
So, the data model is the first axis, and the distribution is the second one.
It’s not trivial to release a distributed relational database. The reason is that distributed transaction is one of the most complex problems in IT, and one SQL operator can involve all the nodes in a single transaction.
There are attractive efforts to create a distributed relational database. You should pay attention to Cockroach or Yugabyte. But these platforms haven’t got widespread.
One day a man invented the in-memory cache. As random access memory got cheaper, in-memory technologies came to databases. Every considered class of platforms contains at least one in-memory member. TimesTen and SolidDB are relational and monolithic; Tarantool, Ignite, etc. are key-value and distributed; VoltDB is relational and distributed.
Now the storage environment becomes the third axis.
You can remember Teradata, Greenplum, MS PDW, and a few more distributed relational platforms. They are very successful commercial software. It’s true, but these platforms are not intended to process transactions.
So the fourth axis is the load type: OLTP vs. OLAP.
I would like to draw a 4-dimension cube on the blackboard, but I can’t :)
There are no clear borders between the described classes. Relational databases get some non-relational facilities, while non-relational platforms implement SQL. Disk-based systems become in-memory features, while in-memory databases learn to store data on disk. Monolithic platforms become distributed versions.
The main idea of this presentation is the following: you have first to define the class of platforms for your solution and then choose a platform inside a class.
Not all the classes are equal. Monolithic platforms are much more robust than distributed ones. Relational model is universal in contrast to NoSQL. On-disk storage is cheaper than in-memory.
That’s why a relational monolithic on-disk platform is almoast always the right choice. So, choose PostgreSQL! This platform really covers more than 90% of problems.

Working on built-in fault tolerance in Postgres

Traditionally, fault tolerance in Postgres is implemented using built-in replication mechanisms and external utilities that monitor the state of running Postgres instances and react accordingly when various failures occur. In this presentation, I will tell you what we like and what we don’t like about this approach, which alternative we see, what we have been able to achieve to date and what we want to get done by the time of release, which is planned for December.

Real-time Query Replanning of Hopeless Queries

During the years of its existence, Postgres Pro piled up a pool of problems when query execution was inappropriately slow or a query was too expensive to be executed, so it was never executed. Almost always in our practice, this was due to the choice of a non-optimal query plan. In our story, we will talk about a very unconventional attempt to solve this problem by re-planning queries. We will tell you what it is, how it works, who will find it helpful and the prospects for using this feature.

VTB-Debezium - a reliable solution for data replication

Facing the restrictions on the use of Western software, our bank has developed and put into commercial operation a reliable mechanism for data replication between two DBMSs. This mechanism allowed the bank's teams without any special restrictions to replace the product from Oracle (Oracle Golden Gate was previously used in the bank for a long time). In the presentation, we will talk about the main advantages of our software and the nuances of its use.

Backups: types and use cases

Backup is still a stumbling block when migrating to PostgreSQL from other DBMSs. Its size directly depends on your experience and knowledge about the types of backups in PostgreSQL. In this talk, I will tell you about different types of backups, their pros and cons, and scenarios for using each type.

Let's take a look at your tables via a microscope

As you probably know, PostgreSQL has a number of distinct features compared to other DBMSs. For example, Postgres can process and store many different types of data. However, you need to know something about them before using them. In this talk, we will find the reason why queries to the table begin to slow down (and autovacuum / vacuum has nothing to do with it) and try to speed up such queries. I will tell you how integer data types work in PostgreSQL and touch on the topic of speeding up such queries. And finally, let's talk about how to make your data in tables take up less space while increasing the speed of queries to this data.

A deep dive into b-tree indices for seasoned application developers

A talk on how regular™ indexes work in PostgreSQL. The talk will be extremely useful for those who are starting to work with databases, and for those who have worked with them but have forgotten. If the talk hits the golden picks of "must-see onboarding developer", then the goal is achieved.
We will tackle the following subtopics:
- How an index can speed up search;
- Should we index WHERE conditions;
- Should we index ORDER BY statements;
- Should we index foreign keys;
- What to do if search pre-assumes multiple conditions;
- Which column order should be specified in an index;
- Cases when index causes a slow-down and how to reduce its impact on the app.

Fuzzing research on PostgreSQL. How we performed it and what was found

Fuzzing research is feeding random input data to a program (or a part of it) (in fact, randomness is very conditional) and seeing what we get out of it. And we repeat it many times on many processors.

Fuzzing a large monolithic program complex is never a simple task. It requires extraordinary solutions. In this talk, I will tell you what and how we searched with the help of fuzzing and what results it led to.

• Investigation of data type parsing functions (input-functions): for warming up;
• Investigation of functions implementing operations between types (op-functions): it is better to consider the structure here;
• Network subsystem fuzzing: let's pretend we are POSIX calls, it's cheaper that way;
• Recovering disk context: we need Groundhog Day.

A story about funny bugs and ridiculous hand gestures will be included.

CFS and PTRACK in pg_probackup – how to use compression better with incremental backup

We will answer the questions about the importance of compression in backup process. Will show a demo how to use the following available algorithms: pglz, zlib, zstd, lz4 in different editions of pg_probackup. We'll show you how to make incremental backups and restore even faster with PTRACK and CFS (tablespace compression) in Postgres Pro. The plans for further development will also be announced.

How to manage PostgreSQL accounts in a microservice environment

In this talk, I will explain how you can organize account management in a microservice environment: organization of a role model, authentication via SSO and cross-service authentication.

All secrets of the new version of Cyberbackup for PostgreSQL: working with cluster, data protection, loss prevention and data recovery, including PITR

An important aspect of PostgreSQL database security and reliability is backup practices. The talk will reveal the main techniques and methods used to protect data, prevent loss and successfully recover information in case of human error, failures or emergencies. We will tell how CyberBackup helps to protect PostgreSQL, including configuration in Patroni cluster: we will consider system deployment options, important aspects of backup policy configuration, including full and incremental backups. We will also show how to easily work with backups via mount and do granular restores.

The new Adaptive Query Optimization technique - AQO 2.0

We will present adaptive query optimization techniques and key capabilities of the new version of AQO and SR_PLAN extensions.

How to solve problems in PostgreSQL using debugging and profiling

Unfortunately, ideal computer systems exist only in science fiction books. PostgreSQL is not exception and sometimes problems may occur. I would like to discuss how to correctly (and incorrectly) try to solve a problem, which way to choose, which tool to use.

The talk is of interest to both beginners and experienced users and database administrators.