Building real-time analytics with PostgreSQL
In today's world, operational reporting and real-time analytics are becoming a basic need. There are a huge number of tools, practices, and approaches, which in turn require different expertise and resources. In this talk, I will tell you how to transform your analytics using PostgreSQL. We'll discuss pitfalls when using different schemes. We'll talk about data quality and performance. I expect this talk to be of interest to both beginners and seasoned practitioners with many years of experience (Discussions and questions after the talk will be highly appreciated). The talk outline is as follows. 1. The evolution of reporting - migration from OLTP to OLAP. 2. Data delivery to DWH and related challenges. 3. Scaling architecture in response to growing data volumes. 4. Data quality issues. 5. Maintaining stability with a large number of changes. 6. Different approaches to organizing the work of the DWH team. 7. And, finally, we'll list the challenges we have successfully responded to with various solutions (pgAgent, PGWatch, working with the file system, the new reading of postgresql.conf).
Видео
Другие доклады
-
45 мин
Anastasia Lubennikova Postgres ProfessionalPostgreSQL partitioning. Work In Progress
Native partitioning was introduced in PostgreSQL 10, and every new release comes with more features and optimizations to this area. Yet, there is still room for improvement.
This talk briefly compares the built-in PostgreSQL partitioning with third-party extensions (pg_pathman and pg_partman), to understand what we still don't have in the core. This talk also includes an overview of partitioning-related features that are currently being developed and aimed for PostgreSQL 14.
-
45 мин
Dimitri FontainePostgres Architectures in Production
When using PostgreSQL in production it is important to implement a strategy for High Availability. With a database service, the HA properties apply to both the service itself and of course to the data set.
In this talk we learn how to think about specific HA needs of your production environment and how to achieve your requirements with Open Source tooling when it comes to your database of choice, PostgreSQL. In particular, we dive in many options that could be implemented for Postgres to evolve its offering from being a toolset to being “batteries included”. What does it mean in the context of HA? How to achieve it?
-
22 мин
Alexey Lesovsky Data EgretNoisia - a PostgreSQL incident generator
Noisia is the result of my past attempts to gather all the tools for reproducing various incidents with Postgres.
Noisia is an utility for easy creation of artificial incidents in a Postgres database. This talk will describe the reason for artifical incidents creation, what Noisia does and what is it's possible use. Also I will share the future development roadmap.
-
45 мин
Dmitry Dolgov Zalando SEHow many engineers does it take to make subscripting work?
Recently landed in PostgreSQL, jsonb subscripting support doesn't look as exciting as some other improvements around jsonb. But it's user visible changes are only tip of the iceberg. How many people were involved to make it, and what decisions choices were made? How long did it take, and what are the good/bad ideas to promote a patch? These and few other questions will be our targets in this talk.