title

text

Ronan Dunklau
Ronan Dunklau Dalibo
: December
45 мин

Multicorn: writing FDWs in Python

Multicorn is a generic Foreign Data Wrapper which goal is to simplify development of FDWs by writing them in Python.

We will see:

  • what is an FDW what Multicorn is trying to solve how to use it, with a brief tour of the FDWs shipping with Multicorn.
  • how to write your own FDW in python, including the new 9.5 IMPORT FOREIGN SCHEMA api.
  • the internals: what Multicorn is doing for you behind the scenes, and what it doesn't

After a presentation of FDWs in general, and what the Multicorn extension really is, we will take a look at some of the FDWs bundled with Multicorn.

Then, a complete tour of the Multicorn API will teach you how to write a FDW in python, including the following features:

  • using the table definition
  • WHERE clauses push-down
  • output columns restrictions
  • influencing the planner
  • writing to a foreign table
  • IMPORT FOREIGN SCHEMA
  • ORDER BY clauses pushdown
  • transaction management

This will be a hands-on explanation, with code snippets allowing you to build your own FDW in python from scratch.

Слайды

Видео

Другие доклады

  • Jean-Paul Argudo
    Jean-Paul Argudo Dalibo

    Migration to PostgreSQL : reasons... and consequences

    The talk will be articulated around all the traditional arguments to "how chose PostgreSQL over other choices in the database domain"... But also, and that's quite new in the comunity, what are the consequences of this choice. Because the PostgreSQL adoption brings adoption of other things like Linux, but also, Open Source thinking, the fast pace of PostgreSQL will command new methods of validation the company must adapt to... etc.

  • Boris Veryugin
    Boris Veryugin ООО "Диасофт Платформа"
    45 мин

    Automated migration of applications from proprietary DBMS to PostgreSQL

    Diasoft Platform company's technology solutions for migration of applications from proprietary DBMS (on the examples of Oracle and Microsoft SQL Server) to PostgreSQL will be presented in the talk. These solutions are implemented in Diasoft Database Adapter software.

    Our technology solutions allow to automate: 1) migration of database schema (including translation of stored procedure and function code); 2) migration of data; 3) migration of client applications without any change in their source code.

  • D
    Dennis Ivanov 2ГИС
    22 мин

    Evolution of PostgreSQL usage in 2gis directory API

    • First aquaintance
    • Fight with replication
    • Partitioning and migration
    • Cross data-center use
    • v8, json, jsonb, jsquery
    • Version upgrade

  • Alexander Korotkov
    Alexander Korotkov Postgres Professional
    45 мин

    PostgreSQL extendability: Origins and new horizons

    Postgres was initially designed to support access methods extendability. Well known citation about access method in Postgres claims: "It is imperative that a user be able to construct new access methods to provide efficient access to instances of nontraditional base types" Michael Stonebraker, Jeff Anton, Michael Hirohama. Extendability in POSTGRES, IEEE Data Eng. Bull. 10 (2) pp.16-23, 1987

    Initially, heap was just one for access methods. So, extendability of access methods would also mean pluggable storage engines in modern terms. For now, only index access methods are defined in pg_am table of system catalog. Those index access methods also have well-defined interface. Therefore in order to meet initial design PostgreSQL need to support two features:

    • Pluggable index access methods, i.e. ability to implement new index types by adding new tuples to pg_am;
    • Pluggable storage engines, i.e. ability to implement completely different storages for tables without traditional heap.

    Besides mechanical work like "CREATE ACCESS METHOD" command, extensible index access methods needs to be WAL-logged. For now, community doesn't want extensions to define their own WAL-records, because there is a chance to break both recovery and replication, which is not acceptable. Another approach is to define generic WAL-records, that specify a difference between pages in generalized way.

    There are only few DBMS which support pluggable storage engines now. MySQL is the most common example here. However, dealing with different storage engines in MySQL is like dealing with different DBMS. This is not the way PostgreSQL should go from our view.

    However, now PostgreSQL users realize benefits from other storages. Ideas of columnar storages and in-memory storages for PostgreSQL are very popular. Simultaneously, technical possibilities to implement them are growing. FDW and custom nodes are arrived. Generic WAL and extensible index access methods are pending for 9.6. Much work in the direction of pluggable storage engines is already done even if it had different aims.

    It's time for PostgreSQL core developers to think about native support of pluggable storages without kludges. Finally, we should get "CREATE STORAGE ENGINE name ..." command as legal extendability mechanism.

    In this talk we will show current state on pluggable index access method and design of pluggable storage engines.