title

text

Andrey Chibuk
Andrey Chibuk ООО "ФОРС Телеком"
Alexander Liubushkin
Alexander Liubushkin ООО "ФОРС Телеком"
10:50 04 April
45 мин

How to transfer 10TB from Oracle to Postgres in 24 hours?

We offer to your attention our experience in data migration and the Ora2PgCopy program written in Java for high-speed data transfer from Oracle to Postgres, which is used after creating tables and transferring the program code of application systems. High data transfer speed is provided by using the Postgres command “copy”, using multithreaded Java technology for file processing, managing the nologged/logged table option, and supporting LOB and CLOB data types. According to the test results, Ora2PgCopy works noticeably faster than such analogues as: Ispirer (convertum), oracle_fdw, ora2pg, Pentaho kettle. Ora2PgCopy can function as a module as part of the LUI4ORA2PG migration automation system or independently of it. The history of the growth of the Live Universal Interface (LUI) web application development tool and the LUI4ORA2PG migration tool can be found in previous presentations at PGConf conferences: https://pgconf.ru/2019/118109 , https://pgconf.ru/201911/264095 , https://pgconf.ru/2020/262456, https://pgconf.ru/2021/288310, https://pgconf.ru/2022/316022.

Слайды

Любушкин-Чибук.ppsx

Видео

Видео доступно участникам мероприятия, выполнившим вход в личный кабинет

Другие доклады

  • Christopher Travers
    Christopher Travers Independent Community Member
    45 мин

    Crazy things you can do with PostgreSQL Indexes

    Of the relational databases, PostgreSQL is fairly unique in the indexing capabilities it offers. While most of us are familiar with the use of indexes to speed performance when filtering on columns, PostgreSQL indexes can do far more than this.

    This talk will focus on cases where difficult problems were solved through the creative use of indexes. Each of these cases is from an episode in my career.

    In this presentation, you will learn:

    • The general index access methods PostgreSQL supports out of the box
    • How the planner uses indexes (only on a high level)
    • Functional indexes
    • Creative use of Index Only scans.

    This is a talk for all audiences. The talk is primarily for beginner to intermediate users, but should have enough information for more advanced users to get some insight or inspiration from the talk as well.

  • Anton Nemtsev
    Anton Nemtsev ООО "Нетрика"
    22 мин

    Case of moving a DBMS with business logic to Postgres PRO

    I will share the experience of migration project of an information system from a Microsoft SQL Server DBMS to Postgres PRO. The owner of the DBMS is Rosagroleasing JSC.

    The volume of the DBMS database is about 6 TB (more than 300 tables, up to 4 billion records in each). The team adapted 15 ETL packages and 131 procedures with full functionality and also partial refactoring of the source code. Systems' performance has remained the same.

  • Christopher Travers
    Christopher Travers Independent Community Member
    45 мин

    PostgreSQL vs Redis: Making the Right Choice

    With the rise of NoSQL databases, a number of falsehoods have flourished regarding how to choose a database engine. This talk focuses specifically on Redis and PostgreSQL, and why one might choose one or the other.

    At small scales, we can often get by thinking of database servers as black boxes, but as we scale, the internals and architecture become more and more important. This talk focuses on behavior of these systems at scale and under load.

    In this presentation you will learn:

    • How Redis and PostgreSQL differ architecturally
    • How differences in architecture affect scalability and performance
    • Cases here Redis is the clear winner
    • Cases where PostgreSQL is the clear winner

    Additionally, some notes will be offered in terms of where PostgreSQL can improve in to compete with the sorts of workloads that generally favor Redis.

  • Bruce Momjian
    Bruce Momjian EnterpriseDB
    45 мин

    Beyond Joins and Indexes

    My presentation "Explaining the Postgres Query Optimizer" covers the details of query optimization, optimizer statistics, joins, and indexes. This talk covers 40 other operations the optimizer can choose to handle complex queries, large data sets, and to enhance performance. These include merge append, gather, memoize, and hash aggregate. It explains their purpose and shows queries that can generate these operations.

    This is a new talk; draft slides are at https://momjian.us/main/writings/pgsql/beyond.pdf