PgConf.Russia 2020

This talk discusses the open source components we use at Adjust to manage a massive number (5+PB) of application log messages on PostgreSQL in a massively multi-parallel way. It provides both a use case for PostgreSQL in a big data (high volume/velocity/variety) environment, and can be used to show the power of PostgreSQL with JSONB, GIN, and more.

This talk covers the capabilities of the components in depth, sufficient to inspire similar solutions.

It is our experience of the medical information system "RT MIS" transfer from MSSQL to PostgreSQL . When the necessity of transfer to PostgreSQL in our "RT MIS", one of the largest medical information systems, became imminent, we felt really terrified having assessed its amount: there was a huge number of stored procedures, functions, SQL-queries in its application code and services. It all requested transcribing, was exacerbated by demands on the system accessibility. So the variant "we awoke in the morning and PostgreSQL was working everywhere" was definitely impossible. That is why we chose another way: began eating "the elephant (PostgreSQL)" in chunks.

In my report, I am going to share our practical experience of the transfer, the instruments we used, the reason for another replication, the problems we met and their solutions. And finally, what turned out to be better: PostgreSQL or MSSQL.

Last year I made a talk about unexpected PostgreSQL bottlenecks, which could make sad surprise to user (or DBA). Feedback to my talk was very positive. Additionally I have new material after year. This is why I'm making a sequel including new unexpected situations when your database hangs. This time focus will be on multicore hardware platforms, but not only them.

Sometimes it maybe a very tricky just to establish a connection with your remote database. Especially if your working place is in highly secured corporate environment and your database server is in highly secured remote cloud.

We will talk about bypassing SSH, SSL, jump servers, proxy servers, VPNs and various SSO (Kerberos/SSPI/LDAP/Active Directory) authentication systems in order to access your PostgreSQL database using console or rich UI interface and JDBC. We will describe different use cases - from trivial and typical to insane and over-secured.