How-to obfuscate Postgres database for load testing in web apps
Postgres is a well-known database for high load web applications. Such apps require stress/load testing itself to run properly in production. Besides obvious difficulties in preparation a test environment identical to production, generating proper traffic there is another one issue - database preparation for the test environment. And it seems it is not good to use the database from production in the testing environment in the era of personal data protection (GDPR, HIPAA). Data obfuscation is the rescue.
There are few instruments for data obfuscation in Postgres. During this session, I will tell you which of them we've selected and why what type of issues we faced, and if our solution was successful. You will know if it is possible to get an identical response on the test database without real data from production, we will observe some restriction on obfuscation, I'll present our utility which simplify things.
Материалы к докладу
Видео
Другие доклады
-
Andrey Borodin ЯндексEvgeniy Dyukov Yandex
How to manage an open source HA RDBMS in a cloud environment
High availability solutions have become extremely popular in the past few years. They play a critical role in building reliable systems based on affordable hardware. In this presentation, we will pay attention to some of the subtle aspects of the design and maintenance of such systems. In addition, the issues of capturing changes on a HA cluster will be addressed.
-
Andreas Scherbaum Pivotal
Managing PostgreSQL with Ansible
Ansible is an open-source configuration management and deployment tool, which can be used to manage servers and software installations. This talk will briefly cover Ansible itself, and then explain how Ansible is used to install and configure PostgreSQL on a server. Examples will round up the talk.
-
Robert Bernier Percona
pg_upgrade, Advanced Techniques
When it comes to performing postgres upgrades between major versions the command line utility pg_upgrade is the most popular method used today. But as with all things wonderful, there are caveats. One of the more critical issues is what to do when there's a failure. The purpose of this talk is to present those little covered "tricks" of the trade that allows one to improve and enhance the upgrade experience.
We'll start out by reviewing its basic mode operation. We'll then learn what makes it so darn fast when upgrading multi-terabyte datacluster in a matter of minutes. Finally, you will be introduced to those feared failures and how to deal with them with confidence and certainty.
Herein is a summary of the topics in this talk:
- How pg_upgrade works: the big picture
- About pgupgrade (comand line invocation)
- arguments and options
- The steps performing an upgrade
- About the replication ROLE
- with replication
- with login
- Upgrade options: COPY vs Hard Link
- Post-upgrade
- about performance
- aboutanalyze
- repack
- reindexing
- When something goes wrong at the point of no return (or is it?)
- Upgrading the REPLICA
- default method: pg_basebackup
- the cool method:
- - leveraging rsync
caveat; about vacuum wraparound
-
Иван Чувашов ООО Calltouch
Life of DBA in the OKKO online cinema
Okko is one of the largest legal online cinemas in Russia. Our catalog contains 60 000 movies, cartoons and TV series. Since its starting, the service has been visited by over 20 million users. The monthly audience is 2.8 million people. All these figures speak of a reliable high-load service.
As DBA, I will mainly talk about databases (PostgreSQL, Cassandra, Redis) that are used in the company. We'll take a closer look at PostgreSQL on the topics of high loads, monitoring, optimization, backup and recovery.