MVCC in pictures and when long transactions create problems
Many of us know that it is MVCC that provides concurrency access to data in many relational databases that guarantee transactions consistency and isolation. But only deep understanding of the implementation of this mechanism in PostgreSQL allows us to better understand the processes in the database, to design the application logic and table structures to be the most effective in a high-load world. We'll take one of the processes in our product to understand how MVCC is implemented in PostgreSQL and we'll explain one of the peculiarity when seemingly unrelated activities can affect each other.
Слайды
Видео
Другие доклады
-
Alexey Klyukin Zalando SEAlexander Kukushkin Zalando SE
Tutorial: Management of High-Availability PostgreSQL clusters with Patroni
Patroni is a Python application to create high-availability PostgreSQL clusters based on the streaming replication. It is used by Red Hat, IBM Compose, Zalando and many other companies. This tutorial will highlight Patroni architecture, provide attendees with hands-on experience of configuring high-availability PostgreSQL clusters with Patroni, describe how to take advantage of numerous additional features and give an opportunity to learn more about common mistakes related to running Patroni and its troubleshooting.
In order to take most out of the Patroni tutorial one needs a laptop with git, vagrant and virtual box installed.
Vagrant can be obtained from https://www.vagrantup.com Virtualbox is at https://www.vagrantup.com
Alternatively, one can install your Linux distribution packages (or use homebrew on Mac).
Once Vagrant and Virtualbox are installed one can run the Patroni VM by issuing the following commands:
$ git clone https://github.com/alexeyklyukin/patroni-training $ cd patroni-training $ vagrant up
When the setup concludes Patroni box can be accessed via ssh using vagrant ssh command.
-
Konstantin Evteev X5 FoodTechMikhail Tyurin ИТ предприниматель
Recovery use cases for Logical Replication in PostgreSQL 10
Avito is the biggest classified site of Russia, and the third largest classified site in the world (after Craigslist of USA and 58.com of China). In Avito, ads are stored in PostgreSQL databases. At the same time, for many years already the logical replication is actively used. With its help, the following issues are successfully solved: the growth of data volume and growth of number of requests to it, the scaling and distribution of the load, the delivery of data to the DWH and the search subsystems, inter-base and internetwork data synchronization etc. But nothing happens "for free" - at the output we have a complex distributed system. Hardware failures can happen - it is natural - you need to be always ready for it. There is plenty of samples of logical replication configuration and lots of success stories about using it. But with all this documentation there is nothing about samples of the recovery after crashes and data corruptions, moreover there are no ready-made tools for it. Over the years of constantly using PgQ replication, we have gained extensive experience, rethought a lot, implemented our own add-ins and extensions to restore and synchronize data after crashes in distributed data processing systems. In this report, we would like to show how our experience can be shifted to a new logical replication subsystem in 10th version of PostgreSQL. In the current implementation, these are only non-trivial solutions - there is a number of issues for the community, that come down to implementing simple recovery mechanisms - as simple as configuring the replication in 10th version.
-
Gregory Smolkin Ozon
-
Ivan Panchenko Postgres Professional
Full text search from A to Ω
Tutorial on Full Text Seach in PostgreSQL, containing all recent improvemets. All recipies necessary for building an application will be given: dictionary and parser configuration, faceted search, fuzzy search, multilanguage search, ranking etc. Participants will be provided with a test database for exercises.