Scaling Reads — Cross-Cutting Pattern

The Problem

Every system eventually becomes read-heavy. The naive response is to throw caches at it, but every read optimization is a consistency debt. Staff engineers treat read scaling as a consistency budget allocation — deciding where, how long, and for whom stale data is acceptable — not as a performance trick.

Playbooks That Use This Pattern

• Distributed Caching — Cache-aside, write-through, invalidation
• CDN & Edge Caching — Edge layer for static and semi-static content
• Search & Indexing — Read-heavy query paths at scale
• Database Sharding — Partitioned read distribution
• Feed Generation — Pre-computed feed reads

The Core Tradeoff

Staff Default Position

Start with read replicas and cache-aside. These are well-understood, operationally simple, and cover 80% of read scaling needs. Add CDN for static or semi-static content where staleness tolerance is measured in minutes. Reach for CQRS or search indexes only when the read model diverges structurally from the write model — not as a performance optimization.

When to Deviate

• Strong consistency required on reads — financial balances, inventory counts, access control checks. No caching layer; read from primary or use synchronous replication with read-your-writes guarantees.
• Write-heavy workload masquerading as read-heavy — if the bottleneck is actually write contention, adding read replicas solves nothing. Profile before scaling.
• Tiny dataset, high QPS — if the working set fits in a single node's memory, a local in-process cache beats a distributed cache. Don't add network hops for data that fits in RAM.
• Multi-region with strict ordering — CDN and replicas introduce cross-region lag. If users see their own writes inconsistently across regions, the read layer is the problem, not the solution.

Common Interview Mistakes

Quick Reference

Staff Sentence Templates

Implementation Deep Dive

Cache-Aside with Redis — The Staff Default

Cache-aside is the dominant read-scaling pattern because it is simple, explicit, and gives the application full control over what gets cached, for how long, and when it gets invalidated. Here is how to implement it properly.

Application-Level Pseudocode

function getUser(userId):
    cacheKey = "user:" + userId

    # Step 1 — Check cache
    cached = redis.GET(cacheKey)
    if cached != null:
        metrics.increment("cache.hit", tags=["prefix:user"])
        return deserialize(cached)

    # Step 2 — Cache miss: fetch from DB
    metrics.increment("cache.miss", tags=["prefix:user"])
    user = db.query("SELECT * FROM users WHERE id = ?", userId)
    if user == null:
        # Cache negative result to prevent repeated DB lookups
        redis.SET(cacheKey, "NULL", EX=60)
        return null

    # Step 3 — Populate cache
    redis.SET(cacheKey, serialize(user), EX=3600)
    return user

Why this order matters. The application reads from cache first, falls through to DB on miss, and writes back to cache after. The cache never writes to the DB — the write path is completely separate. This means a cache failure degrades to "all reads hit DB" rather than "writes are lost."

Redis Commands in Practice

🎯 Staff Move: Use MGET for batch lookups instead of looping GET. Fetching 100 keys in one round-trip takes ~1ms; 100 sequential GETs take ~50ms on a typical network.

Connection Pooling Configuration

Redis is single-threaded, so connection count matters less than connection reuse:

For 20 app servers with pool size 15, you have 300 total connections to Redis. A single Redis 7 node handles ~10K connections comfortably, so this leaves headroom for growth.

Thundering Herd Protection — Singleflight Pattern

When a popular cache key expires, hundreds of requests simultaneously hit the DB for the same row. The singleflight (request coalescing) pattern ensures only one request fetches from DB while others wait for its result:

# In-process lock map (per app instance)
inflight = ConcurrentMap<String, Future>

function getUserCoalesced(userId):
    cacheKey = "user:" + userId

    cached = redis.GET(cacheKey)
    if cached != null:
        return deserialize(cached)

    # Check if another request is already fetching this key
    if inflight.containsKey(cacheKey):
        return inflight.get(cacheKey).await()  # Wait for in-flight request

    # This request wins — it will fetch from DB
    future = new Future()
    inflight.put(cacheKey, future)

    try:
        user = db.query("SELECT * FROM users WHERE id = ?", userId)
        redis.SET(cacheKey, serialize(user), EX=3600)
        future.complete(user)           # Broadcast result to all waiters
        return user
    finally:
        inflight.remove(cacheKey)

🎯 Staff Move: Singleflight is per-process. In a 20-server fleet, up to 20 DB queries fire simultaneously on a cold key. If that is still too many, use a distributed lock (SET key:lock NX EX 5) to coalesce across the fleet — but the added complexity is rarely justified for cache misses.

Monitoring: What to Track

The numbers that matter. For 10K QPS reads, a 95% hit rate means 500 DB reads/sec. A 99% hit rate means 100. That is a 5x difference in DB load from a 4-percentage-point change. This is why hit rate is the single most important cache metric — small improvements have outsized impact on backend pressure.

🎯 Staff Move: Track hit rate per key prefix, not globally. A 97% aggregate hit rate can hide a 40% hit rate on your most expensive query pattern. Break out user:, product:, feed: separately.

Architecture Diagram

Read path (numbered): App server checks Redis (1). On miss, reads from a read replica (2). Writes result back to Redis (3). CDN handles static content before it reaches the application tier.

Write path (dashed): Writes go to the primary DB. The write service issues DEL on the cache key immediately after commit. Subsequent reads re-fill the cache from the replica.

🎯 Staff Move: Mention the three-tier cache hierarchy in interviews: in-process LRU (sub-microsecond, tiny capacity) → Redis (sub-millisecond, shared across fleet) → read replica (millisecond, full dataset). Each tier has a different staleness budget and failure mode.

Failure Scenarios

1. Thundering Herd on Cache Cold Start

Timeline: Redis cluster restarts after maintenance. All cache keys are gone. First requests all miss cache simultaneously. DB receives 50x normal read traffic in the first 30 seconds.

Blast radius: Database connection pool exhaustion → cascading timeouts → 500 errors for all users, not just those with cold keys.

Detection: Cache hit rate drops to 0%. DB active connections spike to pool max. Error rate exceeds threshold.

Recovery:

1. Enable singleflight coalescing to limit concurrent DB fills per key
2. Use cache warming: pre-populate top 1,000 hot keys from a scheduled job before shifting traffic
3. Apply request rate limiting per user to cap the fill rate
4. If DB is overwhelmed, enable a circuit breaker that returns stale data from a backup source

🎯 Staff Insight: A cold cache is not a performance problem — it is an availability problem. Warm the cache as part of the deployment runbook, not as an afterthought.

2. Cache Inconsistency After Failed Invalidation

Timeline: Write service updates a user's email in PostgreSQL, then calls DEL user:12345 on Redis. The Redis DEL fails (network timeout). Cache still holds the old email. User sees stale data for up to the remaining TTL (potentially hours).

Blast radius: One user sees stale data. If the failed invalidation affects a shared entity (product price, feature flag), all users see stale data.

Detection: Hard to detect proactively. Often found via user complaint. Can be detected by comparing cache values to DB values in a background audit job.

Recovery:

1. Retry invalidation with exponential backoff (fire-and-forget is insufficient for critical keys)
2. Use a CDC (change data capture) stream as a secondary invalidation channel — Debezium watches the binlog and issues Redis DEL for every changed row
3. TTL provides a bounded staleness window — stale data eventually expires even if explicit invalidation fails

🎯 Staff Insight: Never rely on a single invalidation path. The write service DEL handles the happy path; CDC handles the failure path. TTL is the backstop.

3. Read Replica Lag Causing Stale Business Decisions

Timeline: Primary DB processes a payment that sets order.status = 'paid'. The read replica has 800ms replication lag. A read request hits the replica and sees order.status = 'pending'. The system triggers a duplicate payment reminder or blocks the user from proceeding.

Blast radius: Any business logic reading from replicas during the lag window makes decisions on stale state. Payment double-charges, inventory oversells, access control violations are all possible.

Detection: Monitor pg_stat_replication.replay_lag. Alert when lag exceeds your staleness budget (e.g., >500ms for order status reads).

Recovery:

1. For critical reads (payment status, inventory counts, access control), always read from primary
2. Implement read-your-writes consistency: after a write, the response includes a monotonic position marker; subsequent reads include this marker and are routed to primary if the replica has not caught up
3. Accept replica lag for non-critical reads (profile display, feed content, analytics)

🎯 Staff Insight: Classify every read path as "critical" or "best-effort." Critical reads go to primary; best-effort reads go to replicas. This classification is a product decision, not an engineering decision.

Staff Interview Application

How to Introduce This Pattern

Open with the read-to-write ratio to anchor the conversation in data, then state the default approach and the consistency tradeoff. This tells the interviewer you have a repeatable mental model, not just a grab-bag of techniques.

When NOT to Use This Pattern

• Write-heavy workload: If the bottleneck is write throughput, read scaling techniques waste effort. Profile first — the constraint might be lock contention, not read volume.
• Strong consistency required on every read: Financial balances, inventory decrements, access control checks. Caching introduces staleness. Route these reads to the primary.
• Tiny dataset, high QPS: If the entire working set fits in application memory (<1 GB), an in-process LRU cache avoids the network round-trip to Redis entirely. Don't add distributed cache infrastructure for data that fits in RAM.
• Rapidly changing data: If every cache entry is invalidated within seconds of being set, the hit rate will be too low to justify the cache layer. The cache only helps when data lives long enough to be reused.

Follow-Up Questions to Anticipate

Capacity Planning Quick Reference

Sizing a Redis Cache-Aside Layer

Latency Budget Breakdown

Key Numbers Worth Memorizing

Common Pitfalls Checklist