Architecting petabyte-scale analytics by scaling out Postgres on Azure with Citus
The story about powering a 1.5 petabyte analytics application with 2816 cores and 18.7 TB of memory in the Citus cluster at the Microsoft. The Windows team measures the quality of new software builds by scrutinizing 20,000 diagnostic metrics based on data flowing in from 800 million Windows devices. At the same time, the team evaluates feedback from Microsoft engineers who are using pre-release versions of Windows updates. At Microsoft, the Windows diagnostic metrics are displayed on a real-time analytics dashboard called “Release Quality View” (RQV), which helps the internal “ship-room” team assess the quality of the customer experience before each new Windows update is released. Given the importance of Windows for Microsoft’s customers, the RQV analytics dashboard is a critical tool for Windows engineers, program managers, and execs.
Слайды
Другие доклады
-
45 мин
Pavel Stehule freelancer Независимый консультант и разработчикThe performance problems related migrations from PL/SQL to PL/pgSQL
Porting applications from Oracle to Postgres is common work today. Unfortunately it is not without problems. In presentation I'll try to show the basic performance problems related to differences between Oracle and Postgres and PL/SQL and PL/pgSQL.
-
45 мин
Shawn Kim Apposha CEOMake Your PostgreSQL 10x Faster on Cloud in Minutes
Cloud storage has some unique characteristics compared to traditional storage mainly because it is virtualized and controlled by software. One example is that AWS EBS shows higher throughput with larger I/O size up to 256 KiB without hurting latency. Hence, a user can get only about 4 MiB/sec with 1,000 IOPS EBS volume if the I/O request size is 4 KiB, whereas a user can get about 250 MiB/sec if the I/O request size is 256 KiB. This is because EBS consumes one I/O in a given IOPS budget for every I/O request regardless of the I/O size (up to 256 KiB). Unfortunately, PostgreSQL cannot exploit the full potential of cloud storage because PostgreSQL has designed without considering the unique characteristics of cloud storage.
In this talk, I will introduce the AppOS extension that improves the throughput of a write-intensive workload by 10x by transparently making PostgreSQL cloud storage-native. AppOS works like a storage driver that efficiently exploits the characteristics of cloud storage, such as I/O size dependency to storage throughput and latency, atomic write support in cloud block storage, and fast, but non-durable local SSDs. To do this, AppOS comprises a Linux-compatible file I/O stack including virtual file system, page cache, block I/O layer, cloud storage driver. On top of the file I/O stack, syscall module supports registering pre- and post-handler for file I/O-related system calls in order to transparently work without modifying PostgreSQL codes.
I will focus on presenting key use cases and performance results of the AppOS extension after explaining the internals. Specifically, I will show the performance results of OLTP and some batch workloads using standard benchmarking tools like pgbench and sysbench. I will also present performance results and implications on multiple clouds including AWS, GCP, and Azure.
-
45 мин
Jose Cores Finotto Gitlab Inc Staff EngineerManaging PostgreSQL at Gitlab.com
I would like to present the main projects for the evolution of our database, how we execute the administration, the problems and pitfalls we found, and how we solve them,the number and how are the database clusters from Gitlab.com , and what is our planning for the future, sharding, kubernetes... Our environment is in an exponential growth, with millions of users and thousands of requests per second, and we keep our platform stable and scaling. Join our session and discover our how we are doing it!
-
180 мин
Антон Дорошкевич ИнфоСофт Руководитель ИТМастер-класс настройки PostgreSQL для 1С
Вас ждёт увлекательное путешествие по настройкам PostgreSQL. Мы поговорим о том что с 1С не так или наоборот так, что под неё надо специально настраивать СУБД. Обсудим подходы к тестирования скорости работы 1С. Рассмотрим различные варианты схем резервного копирования и обеспечения отказоустойчивости. В процессе мы сравним скорость работы 1С на PostgreSQL настроенном по умолчанию со скоростью работы на настроенном для 1С PostgreSQL. Так же мы создадим реплику PostgreSQL, и переключим на неё Сервер 1С "на живую", посмотрим что увидят пользователи 1С при этой операции. И отдельный блок будет посвящён PostgresPro Enterprise Edition, как использование его приемуществ влияет на скорость работы 1С.