How Tencent uses Postgres-XC for their high volume WeChat payment system
Tencent, Chief PostgreSQL Architect of Tencent Cloud Service
Jasonysli, from Tencent Company, the biggest Internet Company of China.He is the Chief PostgreSQL Architect of Tencent Cloud Service. Team Manager of Tencent PostgreSQL Team which has several experienced software development engineers. Worked in Huawei about postgresql from 2009 to 2013. Developped internal version of PG in Huawei. From 2014 to now, working in Tencent with pgxc. Added new feature to private version of PGXC. His team now is running the biggest PGXC cluster in the world, which contains 29 datanodes and 6 coordinators.
Tencent, based in China, is one of the world's largest companies in the social networking space. This talk discusses how Tencent modified the code of Postgres-XC to meet their internal payment system requirements
- Scale out while minimizing service interruption
With data retention of more than 5 years and tens of millions of new invoices per day, Tencent needs to scale out while queries are still running.
- Optimizing query performance on ordered results of 90 million rows
Queries must be fast. To sort and limit from 90 million rows within few seconds, a special sorting method was created to handle this.
- Handling data skew across nodes
Some WeChat users have much more data than others, causing data skew. We added a special data policy to handle this situation.
4, High Availability across IDCs and cities
With real time invoice querying, real time corruption recovery was required.
Using standby datanodes to provide query only service, for workload optimization
Other issues when handling large amount of transactions
a、disk space reclaim after datanodes scale out
b、archive xlog in standby datanodes
c、exhaustion of network card bandwidth when parallel autovacuum
d、minimize memory use of PostgreSQL connections.