title

text

Nikolay Samokhvalov
Nikolay Samokhvalov Nombox LLC
: December
45 мин

Automated database migration testing

In a heavily-loaded project, any change implies non-zero risks of downtime or performance degradation. We constantly see how system complexity, number of database nodes, deployments per week constantly grow. How grows the level of automatization of various activities in CI/CD pipelines, containers, Kubernetes.

Meanwhile, when we look at the topic of testing of database changes–from trivial index creation to complex, almost "surgical" operations like converting int4 PK to int8 one in a multi-terabyte database under load–here we observe an obvious lag in technology and methodology development. In the best case, the changes are verified and approved visually, and here it all depends on the level of experience and tiredness of the reviewer.

In this talk, we will look at how we (Postgres.ai) solve this problem using our solution, Database Lab:

  • instant provisioning of independent thin clones of multi-terabyte databases, ready to be used for testing,
  • integration with existing CI/CD solutions and workflow,
  • collection of metrics that are the most useful for decision support on whether or not each database migration has to be approved (and even fully automated rejection of the most dangerous actions).

Видео

Другие доклады

  • Anton Doroshkevich
    Anton Doroshkevich ИнфоСофт
    Teodor Sigaev
    Teodor Sigaev Postgres Professional
    45 мин

    1С ERP Platform + Postgres = ...

    CTO Postgres Professional Teodor Sigaev and 1C ERP platform expert Anton Doroshkevich will discuss the existing maintenance issues for 1C ERP on Postgres and their potential solutions.

  • Esteban Zimányi
    Esteban Zimányi ULB
    Mahmoud SAKR
    Mahmoud SAKR université libre de bruxelles
    90 мин

    Managing moving objects data with MobilityDB

    MobilityDB is a moving object database extension to PostgreSQL and PostGIS. It has types and functions for storing an querying geospatial trajectories, as first class citizens. The main type is called tgeompoint (temporal geometry point). It represents a complete movement track of a geometry point, such as a car, a bird, or a person. The function speed(tgeompoint) computes the time varying speed of the object, as a tfloat (temporal float). Similar to these examples, MobilityDB has 6 temporal types, and over 300 functions. As such, it is a function-rich platform for Mobility Data Management.

    In this tutorial you will:

    • learn about moving object databases
    • write MobilityDB SQL queries and explore a database of geospatial trajectories
    • walk through the different type, indexes, and functions of MobilityDB.

  • Robert Haas
    Robert Haas EnterpriseDB
    45 мин

    Avoiding, Detecting, and Recovering From Corruption

    PostgreSQL databases can become corrupted for a variety of reasons, including hardware failure, software failure, and user error. In this talk, I’ll talk about some of my experiences with database corruption. In particular, I’ll mention some of the things which seem to be common causes of database corruption, such as procedural errors taking or restoring backups; some of the ways that database corruption most often manifests when it does occur, such as errors indicating inconsistencies between a table and its indexes or a table and its toast table; and a little bit about techniques that I have seen used to repair databases or recover from corruption, including some experiences with pg_resetxlog. This talk will be based mostly on my experiences working with EnterpriseDB customers; I hope that it will be useful to hackers from the point of view of thinking about possible improvements to PostgreSQL, and to end users from the point of view of helping them avoid, diagnose, and cope with corruption.

  • Sushant Pandey
    Sushant Pandey Microsoft
    Alicja Kucharczyk
    Alicja Kucharczyk Microsoft
    22 мин

    The Story About The Migration

    In this talk we want to present how Microsoft team composed of people from two different teams approached the project and solved the migration issues using ora2pg and was able to prove that Postgres Single Server can perform equally well as Oracle Exadata. We will present our ways of working and also some main technical challenges that we faced including migration of BULK COLLECT’s, hierarchical queries, refcursors and others more complicated Oracle constructs.

    The story about a challenging PoC that proved that Postgres can achieve the same performance as Oracle Exadata. The schema that was migrated wasn’t the simplest one you might see. It was quite the opposite. The code was loaded with dynamic queries, BULK COLLECT’s, nested loops, CONNECT BY statements, global variables and lot of dependencies. Ora2pg did a great job converting the schema but left a lot of work to do manually. Also estimates produced by the tool were highly inaccurate since the logic required not the migration but total re-architecture of the code. In this talk we want to present how Microsoft team composed of people from two different teams approached the project and solved the migration issues using ora2pg and was able to prove that Postgres Single Server can perform equally well as Oracle Exadata. We will present our ways of working and also some main technical challenges that we faced including:

    • How estimates do (not) work
    • How we handled BULK COLLECT’s
    • Why we got rid of refcursors
    • How we got stuck with testing of one the packages and how the help from a friend solved the problem
    • How we handled hierarchical queries and drilling down the hierarchy