Herding elephants: Lessons learned from sharding Postgres at Notion

Herding elephants: Lessons learned from sharding Postgres at Notion

https://www.notion.so/blog/sharding-postgres-at-no... link-img
avatar

Iz članka:

Earlier this year, we took Notion down for five minutes of scheduled maintenance. While our announcement gestured at “increased stability and performance,” behind the scenes was the culmination of months of focused, urgent teamwork: sharding Notion’s PostgreSQL monolith into a horizontally-partitioned database fleet.

If you’ve never sharded a database before, here’s the idea: instead of vertically scaling a database with progressively heftier instances, horizontally scale by partitioning data across multiple databases. Now you can easily spin up additional hosts to accommodate growth. Unfortunately, now your data is in multiple places, so you need to design a system that maximizes performance and consistency in a distributed setting.

We decided to implement our own partitioning scheme and route queries from application logic, an approach known as application-level sharding. During our initial research, we also considered packaged sharding/clustering solutions such as Citus for Postgres or Vitess for MySQL.

Difficult lessons learned
While much of the sharding project captured Notion’s engineering team at its best, there were many decisions we would reconsider in hindsight. Here are a few examples:

1

ULOGUJ SE I DODAJ KOMENTAR


Registruj ili prijavi.

VIDI POPULARNE POSTOVE


avatar Vukašin pre 6 sati
1
avatar Nikola pre 12 sati
avatar Nikola pre 11 sati
1
avatar Nikola pre 14 sati
1
2
avatar Nikola pre 4 dana
1
1
avatar Jovana pre 16 sati
avatar Nikola pre 17 sati
1
1
avatar Nikola pre 15 dana
1
avatar Nikola pre 4 dana
1
avatar Aleksandar pre 4 dana