Fast ETL for PostgreSQL
Implementation of ETL is one of the first tasks that any PostgreSQL user needs to solve. There is a misconception that low-code tools implementing ETL procedures reduce requirements for the user, but due to sacrificing performance and/or flexibility to this goal. The presentation will demonstrate that it is possible to design ETL procedures quickly and easily without losing productivity. We will explain how high speed is ensured, how PostgreSQL features are used and how the combination of PostgreSQL and low-code platforms allows you to get rid of one of the most frequent user pains.
Видео
Видео доступно участникам мероприятия, выполнившим вход в личный кабинет
Другие доклады
-
22 мин
Ivan Frolkov Postgres ProfessionalTemporal types and their usage
During my career, I've seen a lot of code, and very often I faced inaccuracies in handling dates and times. Sometimes the parties got discrepancies in the monthly reports due to such inaccuracies; or daily reports were different for Moscow and SFO, etc. I wouldn't call it a serious problem, but it's annoying and time-consuming. In all cases, such issues occur because of neglectful handling of dates and times. In my presentation, I will discuss how we can avoid it.
-
45 мин
.JPG.180x180.jpg "Pavel Tolmachev")
Pavel Tolmachev Postgres ProfessionalCollapse in query plans. Achieve it and manage it
The more tables are involved with the query, the more difficult it is for the scheduler to choose a suitable execution plan (both time and memory usage increase). How can we "tell" the planner that it is better to connect this pair of tables first, and the rest can be connected later? What if we see that a part of our query can be improved, but the optimizer does not do this. In my presentation, I will talk about managing the order of connections. I will explain how we can influence the formation of a query plan using the standard "vanilla" PostgreSQL methods.
-
45 мин
Alexander Kukushkin Zalando SEImplementing failover of logical replication slots in Patroni
Logical decoding and replication slots introduced in PostgreSQL 9.4 (released in 2014) created a solid foundation for implementing built-in core logical replication in version 10 (released in 2017). Unfortunately, there are a few limitations that make logical replication not very useful in real-world scenarios. Logical decoding currently isn’t supported on the standby server, and PostgreSQL allows creating logical replication slots only on the primary server. Or in other words, logical slots are lost on failover/switchover.
Postgres hackers made many attempts to address the problem, and most of them resulted in not too much success. Although, there is one little function introduced in PostgreSQL 11 that made it possible to implement failover of logical replication slots externally.
In my talk I will tell a story of how Patroni solves the problem of logical replication slots failover without using invasive third-party extensions, dig down into some of the Postgres internals in order to prove why this approach is safe, and finally, we will discuss limitations and potential downsides of this solution.
