Database platform architecture and administrating PostgreSQL in Skype
Most of the main Skype services use a database platform based on PostgreSQL and other open-source technologies, such as Skytools, plProxy, pgBouncer, etc. This platform consists of several hundreds of servers with thousands of databases, which process hundreds of thousands of transactions per second. At the same time, the platform architecture allows its users (applications and their developers) to work with "logical" databases, without any worries about their real “physical” structure.
Our Skype Database Platform team is responsible for the database platform infrastructure. We develop automation systems for various processes that help us ensure service reliability and facilitate development, testing, and deployment of code. In this presentation, I will outline the database platform architecture, review its main components, and tell you about the methods we use in our every-day work to ensure high availability, scalability, replication, fault tolerance, and more.
Слайды
Видео
Другие доклады
-
45 мин
Oleg Ivanov Postgres ProfessionalAdaptive query optimization in PostgreSQL
Query optimization is an important problem, which solution has a great influence on DBMS performance, especially for complex queries. In this talk we consider PostgreSQL query optimizer and specifically cardinality estimation problem for correlated clauses, which is one of the most well-known drawbacks of query optimizers in general. In the talk we propose our solution for this problem which involves machine learning methods and is available for PostgreSQL 9.6 as an extension with a patch. We discuss the experimental evaluation, advantages, disadvantages, and fields of application of the proposed approach as well.
VIDEO
-
22 мин
Oleg Bartunov Postgres ProfessionalPostgreSQL JSON Roadmap
The new ISO/IEC 9075-2:2016 standard specifies JSON data and operations syntax and semantics specifics for SQL. This talk overviews the requirements of this standard, and focuses on the differences between them and the actual implementation of JSON/JSONB in PostgreSQL. Special attention will be paid to JSON Path (XPath analogue), SQL/JSON functions and our plans to make Postgres compliant with this standard.
VIDEO
-
22 мин
Alexander Alekseev Postgres ProfessionalZSON, PostgreSQL extension for transparent JSONB compression
ZSON is a PostgreSQL extension for transparent JSONB compression. Compression is based on a shared dictionary of strings most frequently used in specific JSONB documents (not only keys, but also values, array elements, etc). In some cases ZSON can save half of your disk space and give you about 10% more TPS.
VIDEO
-
45 мин
Vadim Yatsenko ООО Прогресс СофтVery large tables in PostgreSQL. Or how to make 60+ Tb to 10+ Tb
The talk will describe how we have implemented storage of large tables (+1 billion rows per day). The project exists in production 2 years. The total amount of data - 300 Tb (25 PostgreSQL servers * 2 Data Center). I'll tell about mistakes in organization of large tables storage in the initial phase of the project, and how these mistakes were corrected. I'll also talk about how to organize the data rotation and archiving. I voiced questions about what we were missing in PostgreSQL 9.4 out of what appeared in the 9.5 and 9.6. And also, what new features we are waiting for new releases of PostgreSQL.