Talks and tutorials

If you care about Postgres performance, there are a number of hardware acceleration options to help with different use cases. Intel Optane DC persistent memory creates new tier in data hierarchy allowing developers to utilize performance of traditional memory combining with volume and persistency of block storage devices. Unlike traditional DRAM-only in-memory systems, where memory is small, expensive, and volatile, Intel Optane DC persistent memory makes it possible to run larger Postgres databases (terabytes) in memory for higher performance. FPGAs are integrated circuits that can be reprogrammed dynamically to accelerate a specific workload such as SQL execution and data compression. FPGA accelerators extend Postgres with hundreds of SQL reader and writer processes that work in parallel on the FPGA. It’s similar to adding hundreds new cores to boost parallel processing on your server.

Slides

Video

Postgres has always had strong support for relational storage. However, there are many cases where relational storage is either inefficient or overly restrictive. This talk shows the many ways that Postgres has expanded to support non-relational storage, specifically the ability to store and index multiple values, even unrelated ones, in a single database field. Such storage allows for greater efficiency and access simplicity, and can also avoid the negatives of entity-attribute-value (eav) storage. The talk will cover many examples of multiple-value-per-field storage, including arrays, range types, geometry, full text search, xml, json, and records.

Slides

Video

Last year I made a talk about unexpected PostgreSQL bottlenecks, which could make sad surprise to user (or DBA). Feedback to my talk was very positive. Additionally I have new material after year. This is why I'm making a sequel including new unexpected situations when your database hangs. This time focus will be on multicore hardware platforms, but not only them.

Slides

Video

I would like to present the main projects for the evolution of our database, how we execute the administration, the problems and pitfalls we found, and how we solve them,the number and how are the database clusters from Gitlab.com , and what is our planning for the future, sharding, kubernetes... Our environment is in an exponential growth, with millions of users and thousands of requests per second, and we keep our platform stable and scaling. Join our session and discover our how we are doing it!

Slides

Video

We will discuss stored procedures and functions, as well as the code within the database in general. We'll take a look at stored procedures from different angles: as a DBA, a database developer, and a backend developer. We will try to understand the specifics of stored procedures that often go against the common coding guidelines, and discuss best practices for working with stored procedures.
This is mainly a motivational talk for database and backend developers, not an advanced tech talk.

Slides

Video

Any DBA needs some kind of tool for historical workload analyse. Assume once at morning your monitoring team will report of sudden performance degradation at 2-3 a.m., and now you need to investigate this issue. What activities was most resource consuming within that hour? There are several tools for solving this problem, and I'll talk about one very easy and convenient tool - pg_profile. It need only a postgres database and a cron-like tool to run, and it will generate a workload profile report for your database as you need it. Ths report will be a good start point for further investigation.

Slides

Video

In my talk, I will tell how we built a geographically distributed system of personal data storage based on Open Source software and PostgreSQL. The concept of the inCountry business is to provide customers with a ready-to-use infrastructure for personal data storage. Our business customers are ensured that their customer’s personal data is securely stored within their country’s borders. We wrote an API and SDK and built a variety of services. Our system complies with generally accepted security standards (SOC Type 1, Type 2, PCI DSS, etc.). We built our infrastructure with Consul, Nomad, and Vault, used PostgreSQL, ElasticSearch as a storage system, Nginx, Jenkins, Artifactory, other tools to automate management and deployment. We have assembled our development and management teams - DevOps, Security, Monitoring, and DBA. We use both cloud providers and bare-metal servers located in different regions of the world. Development of the system architecture and ensuring the stability of the infrastructure, consistent and secure operation of all its components is the main task facing our teams.

Slides

Slides

Video

PostgreSQL 13 Feature Freeze is scheduled for April 2020. Two more commitfests are still accepting new patches. What we can say about PostgreSQL 13. It's possible that rotation rule will work so that new release wouldn't have as many new features as PostgreSQL 12 have. If even it is do, that would be good evolutionary release with a lot of medium feature and infrastructure changes, which prepares postgres for new leap. In this talk I will overview expected novelties in PostgreSQL 13. It would be more or less accurate, since there would be only one commitfest left, which results are possible to forecast.

Slides

Video

A short overview of the main trends in Postgres development - both for the product itself and the community. What has changed in community goals?

Slides

Video

In this tutorial, we will solve several programming problems using JSON functionality that extends SQL capabilities and virtually turns a relational database into an almost ready-to-use application server. :)
We will learn how to use aggregates, both standard and custom, as well as review possible benefits of JSON recursive structure.

Slides

Video

I like stored procedures - it is great technology. But like any other technologies it allows to write not well optimized code. It is not easy to write optimized code, sql statements in complex large applications. On second hand, there are some tools, that can be used very easily, that can help. Postgres has built-in tracking functions possibility. There are PLProfiler and plpgsql_check. With these tools is easy work to detect slow part of applications.With this knowledge, the fix of performance issue is less magic.

Slides

Video

I'll show an example of solving the problem of searching "similar" texts for one given text in big array using GiST index. The problem itself is not much important, but it is easy to understand. Using this problem as example, I'll show one of many methods of adapting GiST index for custom search problems. Maybe this talk will help you to solve other search problems.

Slides

Video

Sometimes it maybe a very tricky just to establish a connection with your remote database. Especially if your working place is in highly secured corporate environment and your database server is in highly secured remote cloud.

We will talk about bypassing SSH, SSL, jump servers, proxy servers, VPNs and various SSO (Kerberos/SSPI/LDAP/Active Directory) authentication systems in order to access your PostgreSQL database using console or rich UI interface and JDBC. We will describe different use cases - from trivial and typical to insane and over-secured.

Slides

Video

Greenplum is a horizontal scalable database based on PostgreSQL core. It is used for OLAP workloads and a standard task is to quickly load or unload large amounts of data. The external data resources are usually another distributed systems. In this talk I will show how Greenplum can work with external data. What is the architecture and performance of external and foreign tables and how to handle streaming data. What will be changed in the next version. The examples of Kafka and ClickHouse connectors development.

Slides

Video

pg_repack is one of the most popular instruments for removing bloat of tables and indexes in Postgres. In most cases, it works perfectly. But if you use such a feature of Postgres as deferred constraints, using pg_repack becomes more difficult or even impossible. I will talk about how we encountered the problem and will describe some workarounds - from internal instruments of Postgres to a small patch for pg_repack.

Slides

Video

In Zabbix-server 4.4, a new Zabbix-agent has become available. It is written in Golang, has a plug-in structure, keeps a permanent connection and makes it possible to control metrics in runtime. We will make an overview of the Zabbix-agent 2 and talk about how to develop plugins for it, in particular, the PostgreSQL monitoring plugin developed by Postgres Professional in collaboration with Zabbix.

Slides

Slides

Video

Porting applications from Oracle to Postgres is common work today. Unfortunately it is not without problems. In presentation I'll try to show the basic performance problems related to differences between Oracle and Postgres and PL/SQL and PL/pgSQL.

Slides

Video

It is our experience of the medical information system "RT MIS" transfer from MSSQL to PostgreSQL . When the necessity of transfer to PostgreSQL in our "RT MIS", one of the largest medical information systems, became imminent, we felt really terrified having assessed its amount: there was a huge number of stored procedures, functions, SQL-queries in its application code and services. It all requested transcribing, was exacerbated by demands on the system accessibility. So the variant "we awoke in the morning and PostgreSQL was working everywhere" was definitely impossible. That is why we chose another way: began eating "the elephant (PostgreSQL)" in chunks.

In my report, I am going to share our practical experience of the transfer, the instruments we used, the reason for another replication, the problems we met and their solutions. And finally, what turned out to be better: PostgreSQL or MSSQL.

Slides

Video

An enterprise-grade PostgreSQL requires many complementary technologies to the database core: high availability and automated failover, monitoring and alerting, centralized logging, connection pooling, etc. That is, a stack of components around PostgreSQL. Kubernetes has enabled a new model to deploy software abstracting away the infrastructure. However, containers are not lightweight VMs, and the packing of software paradigms that work on VMs are not valid on containers/Kubernetes. How should be PostgreSQL and its stack be deployed on Kubernetes? Enter StackGres. An open source software that is the result of re-engineering PostgreSQL to become cloud native. Join this talk to learn and see demos of how to generate PostgreSQL minimal containers; how the sidecar pattern is used (abused) to integrate PostgreSQL’s stack components, and how the networking and storage are handled. More info: stackgres.io.

Slides

Video

Many businesses which use Database management systems like Oracle, DB2 & MS SQL are unreliable these days. Moreover, the costs incurred in maintaining these systems and its product licenses keeps on increasing. As the competitors are migrating over to the new technologies and tools available in the market, it is necessary for these businesses to migrate to new environment which is efficient, consistent and reliable to stay in the market and the technologies used in the current environment have become obsolete or no longer serve the business purpose. PostgreSQL has emerged as a top open-source RDBMS software. Since there is no licensing cost associated with it most of the companies are planning to migrate the databases which are currently running on other RDBMS like Oracle, DB2, MS SQL server to PostgreSQL. This report summarizes the various methodologies, procedures and techniques involved in successfully migrating the data from Oracle to PostgreSQL & DB2 to PostgreSQL. Migration is not a simple effort there should be proper planning and testing involved in this right from database connectivity to performance analysis. In this paper we are going to cover most of the steps which we need to consider before the migration and after the migration like choosing the correct tools for implementing the migration, time taken to migrate ,data compatibility, code conversion, application connectivity to database, database configuration parameters, performance analysis, replication setups, database monitoring, patching and backup strategies.

Slides

Slides

Video

Cloud storage has some unique characteristics compared to traditional storage mainly because it is virtualized and controlled by software. One example is that AWS EBS shows higher throughput with larger I/O size up to 256 KiB without hurting latency. Hence, a user can get only about 4 MiB/sec with 1,000 IOPS EBS volume if the I/O request size is 4 KiB, whereas a user can get about 250 MiB/sec if the I/O request size is 256 KiB. This is because EBS consumes one I/O in a given IOPS budget for every I/O request regardless of the I/O size (up to 256 KiB). Unfortunately, PostgreSQL cannot exploit the full potential of cloud storage because PostgreSQL has designed without considering the unique characteristics of cloud storage.

In this talk, I will introduce the AppOS extension that improves the throughput of a write-intensive workload by 10x by transparently making PostgreSQL cloud storage-native. AppOS works like a storage driver that efficiently exploits the characteristics of cloud storage, such as I/O size dependency to storage throughput and latency, atomic write support in cloud block storage, and fast, but non-durable local SSDs. To do this, AppOS comprises a Linux-compatible file I/O stack including virtual file system, page cache, block I/O layer, cloud storage driver. On top of the file I/O stack, syscall module supports registering pre- and post-handler for file I/O-related system calls in order to transparently work without modifying PostgreSQL codes.

I will focus on presenting key use cases and performance results of the AppOS extension after explaining the internals. Specifically, I will show the performance results of OLTP and some batch workloads using standard benchmarking tools like pgbench and sysbench. I will also present performance results and implications on multiple clouds including AWS, GCP, and Azure.

Slides

Video

Patroni and Stolon are two of the most popular and advanced solutions for orchestrating PostgreSQL and ensuring auto-failover for clusters with the Leader-Followers configuration. However, engineers who migrate from good old tools like Corosync&Pacemaker or built-in solutions of other DBMS often run into issues when installing these tools and figuring out the purpose of each component.
In this tutorial, we will go through a typical installation procedure of Patroni and Stolon clusters on virtual machines (not in containers), as well as review their behavior in case of various infrastructure failures. The whole process will be demonstrated on three vagrant virtual machines with pre-built images. You can follow along on your own system if you set up the required environment.

Slides

Video

200 bases, several clusters, several terabytes of data Share our experience setting up and using patroni cluster DBMS Cluster on Linux, 1C server for windows. We use: PostgreSQL assembly for 1C, Patroni, Consul, Consul dns, Commvault, Ansible Vagrant file and Ansible playbook with roles attached.

Slides

Slides

Video

We would like to share our test results (including performance testing) of PostgreSQL/Citrus and MongoDB for our company's data. It has been a very exciting process with unexpected turns and a somewhat controversial outcome.

Slides

Video

In May 2019 «Postgres Professional» launched the PostgreSQL Certification Program. Now you can not only get training in our courses, but also confirm your knowledge by passing an appropriate set of tests.

In this presentation, I will share the results of the first months of the program and answer the most frequently asked questions:

Where can I get information about the certification program?
PostgreSQL or Postgres Pro certification?
Certificates for PostgreSQL version 10. When will you upgrade to the current version?
Do I have to attend training centers?
Can I pass the test remotely? Not in Moscow?
Is it possible to take several tests at once in one day?
How to prepare for the test? Are there any examples of training questions?
During the test I got an incorrect question, what should I do?
How to find out exactly which questions the answers were not correct?
When can I retake after an unsuccessful attempt?
How to appeal the test results? 

Slides

Video

Role-based access control (RBAC) is one of the main mechanisms used for access control in many DBMS, including PostgreSQL. This model is a sub-type of traditional discretionary access control with its restrictions. In addition to DAC, many operating systems also use mandatory access control (MAC) based on security labels. This additional security mechanism is obligatory for protecting information that demands higher levels of security. Naturally, we would like to use MAC within DBMS when working in OS with mandatory access control switched on.
In this talk, we'll give an overview of existing MAC implementations in DBMS, as well as share our approach to using security mechanisms provided by SELinux, the sepgsql extension for PostgreSQL, and the standard mechanism of row-level security (RLS), which has been available in PostgreSQL starting from version 9.5.
In our presentation, we will use the "Airlines" demo database provided by the Postgres Professional company to show how to protect sensitive information and personal data, compare different ways of storing security labels, and assess performance of our solution.

Slides

Video

I will speak about our experience in solving the fault-tolerance problem for PostgreSQL, which options we reviewed and how we chose Patroni.

I will cover solution testing, quick implementation to the production and issues we faced as well as the way we solved them.

Slides

Video

A brief story how MySQL → PG migration could increase company efficiency tenfold times:

1. Program code has been reduced 50 times, with optimization of backend team (from 15 to 3 engineers)
2. Software development of new features has become measuring in days, not in months
3. Infrastructure costs per 1M users have been reduced 20 times
4. Database structure and technical documentation were simplified significantly, from 100K high-dependent tables to just 20 simple tables
5. New security level because of total forbidden on external SQL commands to the database
6. Quick analytics aggregation on multiple parameters, without external analytics systems
7. The last, but not the least: the main business was keeping alive during migration

Slides

Video

One of the most widespread branches of PostgreSQL distribution in Russia is 1C. In the report I would like to share thoughts about what is missing in PostgreSQL according to 1C

Slides

Video

Will be demonstrated in the master-class the following: 1). The benefits of declarative programming. Rapid creation of complex LUI-forms using only SQL. 2). Support for long transactions and automatic locks. An example of a wizard with a record directly in the database. 3). The multilingual fields and columns. Rapid inclusion of multilingualism in existing forms. 4). Control the coloring of data in fields and columns of the form. Typical examples. 5). Live search in input fields using the example of address input. 6). How to configure the LUI to enter and display of application-specific data. 7). Design form Master Detail with retrieving data from two different databases. 8). How end-user builds of complex data queries without using SQL.

Slides

Video

PostgreSQL is one of the most readily extensible databases in the world. Custom data types, aggregations, functions, and more can be easily and safely written in C.

This hands-on tutorial covers the basics of writing functions, data types, and aggregates in C. It is recommended (though not strictly required) that bring their own laptops and be prepared to actually try the exercises.

Topics covered include: 1. The PostgreSQL type system 2. Creating a simple custom type in C 3. Common errors for non-C programmers 4. Creating a simple aggregate in C

There will be many opportunities for questions and discussion through the tutorial.

Slides

Video