Picture this. You've built an e-commerce backend on PostgreSQL. Everything is on a single server — one instance, one disk, one brain making all the decisions. For the first few months, life is good. Then Diwali sale traffic hits, and your dashboard starts looking unhappy: CPU pinned at 90%, query latency creeping from 50ms to 500ms, connections queuing up.

Your first instinct — and the correct first instinct — is vertical scaling. You go to your cloud console and upgrade the machine:

Before:  4 vCPU  | 16 GB RAM | 500 GB SSD
After:   16 vCPU | 64 GB RAM | 2 TB NVMe SSD

More cores mean more parallel query execution. More RAM means a bigger buffer pool/page cache, so more of your working set stays in memory instead of hitting disk. A faster SSD means lower I/O latency for the queries that do have to touch disk. For a good while, this single lever fixes almost every scaling problem you throw at it. It's also operationally free — no application changes, no new failure modes, no distributed systems to reason about. You just resize the box and reboot (or fail over).

This is why vertical scaling is almost always the right first move. It's cheap in engineering effort, even if it's expensive in hardware cost.

Part 2: The Wall You Eventually Hit

But vertical scaling has a ceiling, and it's not just a budget ceiling — it's a physical one.

• Hardware limits: Even the largest cloud instances top out at some finite number of vCPUs and RAM. You cannot buy a machine with infinite memory.

• Single point of failure: One box means one box can die. Vertical scaling doesn't buy you availability — it just delays the inevitable outage.

• Diminishing returns: A single database instance has one write path, one WAL/redo log, one lock manager. Beyond a point, adding more CPU cores doesn't help because your bottleneck isn't compute anymore — it's contention: rows getting locked, connections queueing, the transaction log becoming a serialization point.

• Cost curve isn't linear: Going from 16 to 32 vCPUs doesn't cost 2x, it often costs 4x or more, because you're now paying for rarer, higher-tier hardware.

At this point, you're not fighting a hardware problem anymore — you're fighting an architectural one. A single node, no matter how big, is still one node. And this is exactly where horizontal scaling enters the conversation.

Part 3: Horizontal Scaling — Spreading the Load Across Machines

Clearing the clutter in the start, What about using Replication?

**Read replicas solve half the problem -**It lets you horizontally scale reads by copying entire data across multiple nodes — but they don't touch the actual bottleneck once your write volume outgrows a single machine, because every replica still funnels through one primary for writes.

The core Idea of Horizontal scaling and how Sharding is related here?

Sharding is the piece that actually solves horizontal write scaling for a SQL database: by physically partitioning the dataset itself, each shard only owns — and only has to write — its own slice of the data but at the cost of complexities like ike cross-shard transactions, broken joins, and painful resharding — trade-offs that turn a simple scaling decision into a genuine distributed systems problem.

The crucial point to keep in mind before moving forward - sharding is the mechanism that actually scales SQL writes horizontally. Read replicas only help with reads; sharding is what splits the write load itself across independent servers.

Horizontal scaling means: instead of making one server bigger, you add more servers and split the work between them.

For reads, this is relatively approachable — you spin up read replicas, and route SELECT-heavy traffic to them via a load balancer while writes still go to the primary. Most teams do this early because it's low-risk.

But writes are the real problem. A read replica doesn't help you if your write volume itself has outgrown a single machine's capacity — one node can only accept so many writes per second before its disk I/O and lock manager become the bottleneck. That's the point where teams reach for sharding: splitting your dataset itself across multiple independent database instances, so that each shard only owns a slice of the data — and, crucially, only handles the write load for that slice.

Part 4: How Sharding Actually Works

Sharding takes one logical table and physically distributes its rows across N independent database servers. Each shard is a full, working SQL database — with its own disk, its own connections, its own query engine — but it only holds a fraction of the total rows.

There are three common strategies for deciding which row goes to which shard:

1. Range-based sharding Rows are split by a range of the shard key — e.g., users A–M go to Shard 1, N–Z go to Shard 2. Simple to reason about, but prone to "hot ranges" (if most of your users have names starting with S, Shard 2 burns).

2. Hash-based sharding You hash the shard key (say, user_id) and use the hash to decide the shard: shard = hash(user_id) % N. This spreads data much more evenly than range-based sharding, since a good hash function scatters keys uniformly regardless of their natural distribution.

3. Directory-based sharding A separate lookup service maintains a mapping of key → shard. More flexible (you can move individual keys), but that lookup service itself becomes a new critical dependency and potential bottleneck.

Here's the mental picture:

                     ┌─────────────────────┐
                     │   Application /      │
                     │   Query Router        │
                     └───────────┬──────────┘
                                 │  shard = hash(user_id) % 3
              ┌──────────────────┼──────────────────┐
              ▼                  ▼                  ▼
       ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
       │   Shard 1    │    │   Shard 2    │    │   Shard 3    │
       │ users 1-33%  │    │ users 34-66% │    │ users 67-100%│
       │ orders, etc. │    │ orders, etc. │    │ orders, etc. │
       └─────────────┘    └─────────────┘    └─────────────┘

On paper, this looks clean. Reality is far messier — and that messiness is the actual point of this blog.

Part 5: The Hero of the Show — Why Horizontal Scaling (Especially Writes) Is Genuinely Hard

This is where most "just shard it" advice quietly skips over the painful parts. Let's not skip them.

5.1 The Cross-Shard Write Problem

In a single-node SQL database, a transaction like "transfer money from Account A to Account B" is trivial — both rows live on the same disk, protected by the same lock manager, wrapped in one ACID transaction.

Now shard by account_id. What if Account A lives on Shard 1 and Account B lives on Shard 2?

BEGIN TRANSACTION
   UPDATE accounts SET balance = balance - 100 WHERE id = A   -- Shard 1
   UPDATE accounts SET balance = balance + 100 WHERE id = B   -- Shard 2
COMMIT

There is no single database engine that can atomically commit this across two independent Postgres/MySQL instances. You've stepped outside the guarantees of a single-node ACID transaction and into the world of distributed transactions.

The classical fix is Two-Phase Commit (2PC):

Coordinator                Shard 1                  Shard 2
    │──── PREPARE ─────────▶│                          │
    │──── PREPARE ────────────────────────────────────▶│
    │◀──── YES/READY ────────│                          │
    │◀──── YES/READY ───────────────────────────────────│
    │──── COMMIT ───────────▶│                          │
    │──── COMMIT ──────────────────────────────────────▶│

2PC works, but it's expensive and fragile:

• It's a blocking protocol — if the coordinator crashes between PREPARE and COMMIT, both shards sit holding locks indefinitely, waiting for a decision that may never come.

• It adds latency — every cross-shard write now needs at least two network round-trips instead of one local commit.

• It reduces availability — every participant must be reachable and healthy for the transaction to complete; one slow or down shard stalls the entire write.

This single problem is the reason "just shard your writes" is much easier said than done.

5.2 Joins and Referential Integrity Break Down

In a single database, JOIN orders ON orders.user_id = users.id is one query, handled entirely within the engine's optimizer.

Once users and orders are sharded independently (or co-located imperfectly), that join has to become one of:

• Scatter-gather: send the query to every shard, then merge results in the application layer. Correct, but slow — you're now bottlenecked by your slowest shard for every such query, and you've moved join logic that the database used to optimize for you into your own code.

• Co-partitioning: deliberately store orders on the same shard as the users row it belongs to (using the same shard key), so the join stays local. This works — but only for that one relationship. The moment you need to join orders with products (which is sharded by product_id, not user_id), you're back to scatter-gather.

Foreign key constraints suffer the same fate — a database can't enforce orders.user_id REFERENCES users.id across two physically separate instances. That referential integrity check now has to be enforced (or, more often, silently not enforced) in application code.

5.3 The Scatter Problem and Partition Pruning

Even for pure reads, if your query doesn't include the shard key, the router has no way to know which shard holds the answer — it has to broadcast the query to every shard and merge results:

Query: SELECT * FROM orders WHERE status = 'pending'
       (no user_id in the WHERE clause)

Router → Shard 1: run query
Router → Shard 2: run query
Router → Shard 3: run query
       ↓             ↓             ↓
     merge all results in the app layer

This is the scatter-gather problem, and it defeats much of the point of sharding — instead of one node doing 1/N of the work, all N nodes do the full query, and you pay N times the total system cost for one logical request. Good sharding schemes are designed so the most frequent queries include the shard key, enabling partition pruning — the router can go straight to the one shard that matters instead of asking everyone.

5.4 Resharding Is Its Own Nightmare

Say you launched with shard = hash(user_id) % 4. Traffic grows, and you need to go to 8 shards. With plain modulo hashing, this changes the destination shard for the vast majority of existing keys:

% 4  → hash(user_id) % 4
% 8  → hash(user_id) % 8

Almost every key maps to a different shard the moment N changes!

That means a near-total data reshuffle — moving the majority of your dataset across the network — while the system is live and still taking writes. This is precisely the problem consistent hashing was invented to solve: shards (and their virtual nodes) sit on a hash ring, and adding or removing a node only remaps the small arc of keys adjacent to it, not the whole ring.

              Hash Ring (0 ────────────────── 2^32-1)
                     ┌───────────┐
              ╭──────┤  Shard A   ├──────╮
             /        └───────────┘       \
      Shard D                              Shard B
             \        ┌───────────┐       /
              ╰──────┤  Shard C   ├──────╯
                     └───────────┘

  Adding Shard E only steals keys from its
  immediate neighbor on the ring — not from all shards.

Even with consistent hashing, virtual nodes (multiple ring positions per physical shard) are needed to avoid uneven load, and the actual data migration — copying rows, keeping both old and new locations consistent mid-migration, and cutting over writes without downtime — is a genuinely hard piece of distributed systems engineering, not a config change.

5.5 Hotspots Don't Go Away — They Just Move

Sharding assumes a roughly even key distribution. Real-world traffic rarely cooperates. A single viral product, a single celebrity user, or a single tenant in a multi-tenant system can dominate the write volume for their one shard, while the other N-1 shards sit idle. You've successfully scaled the average case while leaving the worst case — the one that actually pages you at 2 AM — completely unsolved.

5.6 Putting It Together — Why "Just Shard It" Undersells the Problem

None of this means sharding is a bad idea — for write-heavy systems at real scale, it's often the only path forward. But it's a trade: you're exchanging the simplicity of single-node ACID guarantees for the capacity to scale writes horizontally, and every piece of that simplicity you give up has to be rebuilt, deliberately, somewhere else in your architecture — in your routing layer, your transaction design, or your application code.

Closing Thought

Vertical scaling buys you time. Horizontal scaling — and sharding specifically — buys you capacity, but the invoice comes due in complexity: distributed transactions, broken joins, live data migrations, and hotspot management. Understanding why it's hard is what separates teams that shard successfully from teams that shard and then spend the next year firefighting consistency bugs.

If there's one takeaway to carry forward: don't shard until vertical scaling and read replicas are truly exhausted — and when you do shard, design your shard key around your actual query patterns, not just an even data distribution. The hardest problems above (cross-shard joins, cross-shard transactions) are almost always solvable by choosing a shard key that keeps related data together — the real engineering skill isn't sharding itself, it's picking the right key.

Say you shard your users table across 4 database nodes /shards using a hash function -> hash(user_id) % 4. This works fine — until you add a 5th node. Now % 4 becomes % 5, and almost every single key maps to a different node. You'd have to reshuffle nearly all your data just to add one machine. For a live system, that's a massive, disruptive migration.

Consistent hashing exists to fix exactly this: when you add or remove a node, only a small fraction of keys need to move — not all of them unlike normal hashing.

The core idea: a ring, not a line

Instead of hash(key) % N, consistent hashing imagines the entire output space of your hash function (say, 0 to 2³²−1) bent into a circle — a ring.

• Both your nodes (servers) and your keys (user_ids, order_ids, whatever you're partitioning) get hashed onto positions on this same ring.

• The record gets partitioned when we hash the partition key of the record in a hash function, this gives a unique position to this record on this ring and the nodes are hashed by running a hash function on a unique identifier value of that node like hostname/url etc and the value again places the nodes on the ring.

• To find which node owns a given key: start at the key's position and walk clockwise until you hit the first node. That node owns the key.

• Once the nodes and all records are placed on the ring the very next node towards the clock wise direction to the record is the node where that record lives physically.

That one rule — "clockwise to the nearest node" — is the entire mechanism.

Lets Walkthrough Consistent Hashing with an example scenario to understand it clearly:

Lets say you have following tables:-

1. users

2. orders

3. products

You want to shard the data across 2 servers/nodes.

NOTE: Data partitioning is the logical step where you take a conscious decision on how you want to partition the data based on your use case, load, access pattern (hot keys often used in your system for data querying).

Keeping that note in mind lets first partition our data and understand which shard receives what part of our data.

Step 1: Partitioning users

The very first decision: pick a partitioning key for the users table. Lets chose user_id. This means: take every user's user_id, run it through a hash function, and the output number is that user's position on a conceptual circle (the "ring") going from 0° to 360°.

That's it — that's all "partitioning" means at this stage. No servers involved yet. We've just decided which column determines where a row conceptually sits.

Step 2: Partitioning orders — the same way

For orders, we made a deliberate choice: don't partition by order_id. Instead, partition by the user_id that each order belongs to.

Why?

So that a user's orders land at the exact same ring position as that user themselves — because they're hashing the identical value which is the user_id to hash both the user records and the order records so the output remains same and hence order records of a user gets the exact same position on the ring where that user sits which ensures user records and order records of that user is bound together on the ring. This will also ensure they are in the same partition.

NOTE: This is the exact deliberate logical decision you need to make, here this decision works fine because we got our users and orders grouped into same partitions. This means in a partition/range some user records exist and also all order records related to these users exist with them.

This gets more clear further.

Step 3: Now bring in physical servers — the ring gets nodes

We have 2 physical machines available: call them Server 1 and Server 2. To place them on the ring, we hash their identifiers (hostname, IP — something unique to the machine) through that same hash function. Whatever position that hash produces is where that server sits on the ring.

Say this hashing happens to put:

• Server 1 at the 240° mark

• Server 2 at the 0°/360° mark (the very top, wrapping around)

Now lets visualise this on our ring:-

The two white circles are our servers which are placed on the ring based on a calculated value we got when we run the hash function on a unique identifier value of the servers.

The blue and green arc are nothing but partitions which actually has data - both user and order records. Each position in the ring belongs to either blue or green part which are nothing but our partitions.

So the placing of servers divided the ring into partitions.

In our case what these partitions contain?

Both the green and blue arc contains:-

user records - say green arc has users with user id 10 - 20 and blue arc has users 1 - 10

hence the green arc also contains all orders for users with user id from 10- 20 and blue arc contains all orders for users with user id 1 - 10.

What does this signify?

We have two partitions blue and green and both partitions have a subset of data from users - some user records.

But lets remember step 2 where we made a choice to partition the order records by the user_id, This ensured that a order record which belongs to user id = 1 (or any other user) is placed exactly where the user record with user id = 1 is placed on the ring and ensures that users and orders are partitioned but with all orders of a specific user is bound to that partition itself where the user exists.

Which shard does the data from partition goes?

We have two partitions blue and green and two servers placed at 0°/360° and 240° on the ring.

The ownership rule is clear: A server owns every ring position from the previous server's position (not including it) clockwise up to and including its own position.

Hence all the data in the blue partition goes to the server placed at the point 240° and all the data inside the green partition goes to the server at 0°/360°

With that, you just successfully partitioned the data and also assigned them to shards with a informed design decision to partition and shard your data based on the user_id field.

Why this solves the reshuffling problem

Lets take a similar ring example where we have a few nodes and a few records sitting on the ring:-

The bigger blue circles are our actual physical servers which got their position on the ring by hashing the unique identifier value of those servers and the smaller brown circles represents some records which got their position on the ring by hashing the partition key in those records.

Now suppose Node B goes down, or you add a new Node E somewhere on the ring. Only the keys sitting in the arc immediately counter-clockwise of that one node are affected — they get reassigned to the next node clockwise. Every other key, sitting on other arcs, doesn't move at all. Roughly 1/N of the keys move, not all of them. That's the entire win over hash % N.

Virtual nodes: fixing uneven load

With only 2 real nodes placed on a ring, the arcs between them can be wildly uneven — one node might own 60% of the key space, another 5%. The fix used in practice (Cassandra, DynamoDB all do this): each physical node is hashed onto the ring multiple times under different virtual identifiers — NodeA-1, NodeA-2, NodeA-3... maybe 100–200 virtual points per physical node. This smooths out the arcs so load balances evenly, and it also means when a node fails, its load gets spread across many other nodes instead of dumping it all onto one neighbour.

This is how Consistent Hashing ensures following:-

1. Data is Partitioned and Sharded effectively coupled with correct logical/design thinking.

2. Ensures that roughly 1/N of the total records need to reshuffle instead of all the records where N is the number of servers.