title

text

Andrey Fefelov
Andrey Fefelov Mastery.pro
11:10 02 March
22 мин

How-to obfuscate Postgres database for load testing in web apps

Postgres is a well-known database for high load web applications. Such apps require stress/load testing itself to run properly in production. Besides obvious difficulties in preparation a test environment identical to production, generating proper traffic there is another one issue - database preparation for the test environment. And it seems it is not good to use the database from production in the testing environment in the era of personal data protection (GDPR, HIPAA). Data obfuscation is the rescue.

There are few instruments for data obfuscation in Postgres. During this session, I will tell you which of them we've selected and why what type of issues we faced, and if our solution was successful. You will know if it is possible to get an identical response on the test database without real data from production, we will observe some restriction on obfuscation, I'll present our utility which simplify things.

Материалы к докладу

Видео

Другие доклады

  • Andrey Borodin
    Andrey Borodin Яндекс
    Evgeniy Dyukov
    Evgeniy Dyukov Yandex
    45 мин

    How to manage an open source HA RDBMS in a cloud environment

    High availability solutions have become extremely popular in the past few years. They play a critical role in building reliable systems based on affordable hardware. In this presentation, we will pay attention to some of the subtle aspects of the design and maintenance of such systems. In addition, the issues of capturing changes on a HA cluster will be addressed.

  • Andreas Scherbaum
    Andreas Scherbaum Pivotal
    45 мин

    Managing PostgreSQL with Ansible

    Ansible is an open-source configuration management and deployment tool, which can be used to manage servers and software installations. This talk will briefly cover Ansible itself, and then explain how Ansible is used to install and configure PostgreSQL on a server. Examples will round up the talk.

  • Robert Bernier
    Robert Bernier Percona
    45 мин

    pg_upgrade, Advanced Techniques

    When it comes to performing postgres upgrades between major versions the command line utility pg_upgrade is the most popular method used today. But as with all things wonderful, there are caveats. One of the more critical issues is what to do when there's a failure. The purpose of this talk is to present those little covered "tricks" of the trade that allows one to improve and enhance the upgrade experience.

    We'll start out by reviewing its basic mode operation. We'll then learn what makes it so darn fast when upgrading multi-terabyte datacluster in a matter of minutes. Finally, you will be introduced to those feared failures and how to deal with them with confidence and certainty.

    Herein is a summary of the topics in this talk:

    • How pg_upgrade works: the big picture
    • About pgupgrade (comand line invocation)

      • arguments and options

    • The steps performing an upgrade
    • About the replication ROLE

      • with replication
      • with login

    • Upgrade options: COPY vs Hard Link
    • Post-upgrade

      • about performance
      • aboutanalyze
      • repack
      • reindexing

    • When something goes wrong at the point of no return (or is it?)
    • Upgrading the REPLICA
      • default method: pg_basebackup
      • the cool method:
        • - leveraging rsync
        • caveat; about vacuum wraparound
  • Иван Чувашов
    Иван Чувашов ООО Calltouch
    45 мин

    Life of DBA in the OKKO online cinema

    Okko is one of the largest legal online cinemas in Russia. Our catalog contains 60 000 movies, cartoons and TV series. Since its starting, the service has been visited by over 20 million users. The monthly audience is 2.8 million people. All these figures speak of a reliable high-load service.

    As DBA, I will mainly talk about databases (PostgreSQL, Cassandra, Redis) that are used in the company. We'll take a closer look at PostgreSQL on the topics of high loads, monitoring, optimization, backup and recovery.