Introducing Bagger: Massive Application Log Management on PostgreSQL
This talk discusses the open source components we use at Adjust to manage a massive number (5+PB) of application log messages on PostgreSQL in a massively multi-parallel way. It provides both a use case for PostgreSQL in a big data (high volume/velocity/variety) environment, and can be used to show the power of PostgreSQL with JSONB, GIN, and more.
This talk covers the capabilities of the components in depth, sufficient to inspire similar solutions.
Слайды
Видео
Другие доклады
-
Антон Нечеухин Miro
Tool as code: testing Postgres
At the master class, we will learn how to execute fast load tests of Postgres databases: optimizing database configurations, data structure, indexes, OS settings, etc. To do this, we will create a code, build the infrastructure for the test from it and will do the test. As a result, we get a flexible tool in the code to which you can attach any monitoring, and for which you don't have to pay a lot of money, because the environment is created in 7 minutes in an empty AWS account and destroyed after test
-
Álvaro Hernández OnGres
Kubernetes crash course for Postgres DBAs
Kubernetes is the new way of deploying software, programmatically, on almost any infrastructure (be it cloud or on-prem). But is a complex beast. How to get started? How to dive deeper? What are the specific best-practices and special hints for Postgres DBAs dealing with Kubernetes? Join this half-day tutorial to learn, practically, among other topics:
- How to quickly get started with Kubernetes
- Manage storage
- Manage services, networking and ingress/egress
- How to make Postgres cloud-native in Kubernetes
- Do a show-run of existing Postgres operators, including Zalando, CrunchyData and StackGres.
This tutorial is very practical. BYOL! (Bring Your Own Laptop). With Kubernetes installed! (check microk8s, minikube or k3s if you don’t have any installed.
-
Kamil Islamov Stickeroid Ai
Sequences used for business-logic implementation
Examples of Sequences' opportunities implementations for developing business-logic powered by stored functions.
-
Александр Чирков Барс ГрупАлександр Кварацхелия БАРС Груп
Migration from Oracle to PostgreSQL using an automatic converter
In the report, we want to talk about the experience of migrating one large system from Oracle to PostgreSQL. The system itself was built on the PHP + Oracle stack; its distinguishing feature was that all business logic was implemented in PL / SQL code. In a DBMS, there are more than 3000 packages with 4-10 functions (procedures) in each. In PHP - more than 10,000 forms with inserts of anonymous blocks used to receive data, process and save results in Oracle.
To solve this extremely voluminous work, we took ANTLR4 (a powerful parser generator for reading, processing, executing, or translating structured text), PL/SQL grammar, and created an automatic converter that allows you to convert all objects in the schema and our system from Oracle to working code for PostgreSQL.