{"id":6218,"date":"2019-08-12T16:33:27","date_gmt":"2019-08-12T16:33:27","guid":{"rendered":"https:\/\/gtechbooster.com\/?p=6218"},"modified":"2023-06-21T11:42:55","modified_gmt":"2023-06-21T11:42:55","slug":"paypal-makes-hera-open-source","status":"publish","type":"post","link":"https:\/\/gtechbooster.com\/paypal-makes-hera-open-source\/","title":{"rendered":"PayPal makes Hera Open Source"},"content":{"rendered":"\n

PayPal has made HERA, its H<\/strong>igh E<\/strong>fficiency R<\/strong>eliable A<\/strong>ccess\n to data stores, open source. Hera is a data access gateway that PayPal \nuses to scale database access for hundreds of billions of SQL queries \nper day.<\/p>\n\n\n\n

\"\"<\/a><\/div><\/div>

Hera is made up of a language-specific Hera client library that resides on each application host and allows applications to communicate with Hera efficiently, alongside a centralized Hera proxy service that is fully aware of database configurations and application requirements.<\/p>\n\n\n\n

This service receives application requests and routes them to a chosen database instance. The Router, Parser, & Query Rewriting modules are all part of the Hera proxy. A set of workers that are responsible for maintaining connections to a specific database then handle communication between the proxy and this database. Hera was originally written in C++ but has recently been rewritten in GoLang. <\/p>\n\n\n\n

PayPal developed Hera to scale thousands of its applications with \nconnection multiplexing, read-write split, and sharding. The product was\n developed to handle the problem PayPal found when it moved to using a \nmicroservices architecture in the application tier. This had the side \neffect of creating many application servers and an associated increase \nin pooled DB connections, with large numbers of persistent connections \nin-bound at the DB.<\/p>\n\n\n\n

To overcome the need for persistent connections, PayPal built a \nconnection multiplexer that could multiplex many inbound database \nconnections, most of which were idle, from the application tier to a \nsmall set of active database connections in the DB tier. This was \nimproved on with SQL parsing so it could differentiate between \ntransactional and non-transactional requests.<\/p>\n\n\n\n

Hera also supports a read\/write split to ensure that spikes in the \nnumber of reads don\u2019t affect write performance or vice-versa. It also \nconverts queries from applications to SQL that ensures the query is \ncompatible with sharding. Hera uses sharding to control data \nredistribution. The sharding logic is kept on the server to avoid the \nneed for individual applications to manage sharding on the client side.<\/p>\n\n\n\n

Hera also has a surge queue that deals with high volumes of service \nrequests. At times of high demand, requests can be put into the queue \nand processed as soon as another service request is complete. It can \nalso be configured to remove a slow query to allow multiple more typical\n queries to run.<\/p>\n\n\n\n

Hera’s sharding is based on Oracle RAC clusters. Several RAC clusters\n are used to mimic a single logical database, with logical shards \ndistributed over the RACs and shard routing handled by Hera. Hera uses \nshard key column bind name and value to determine how to direct queries \nto the appropriate shard. The parsing component is used to detect and \nextract the shard key from SQL Where clauses, and the routing component \nroutes queries to a database based on the shard_id extracted by the \nParsing component<\/p>\n\n\n\n

Hera is available on GitHub.<\/p>\n\n\n\n

More Information<\/h2>\n\n\n\n