Caching Strategies & Systems

Caching stores copies of frequently accessed data in fast, low-latency volatile storage (RAM) to dramatically accelerate read performance and protect databases from load spikes.

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

1. Why Caching Matters

2. Caching Strategies & Design Patterns

1. Cache-Aside (Lazy Loading) — Most Popular

The application orchestrates reads and writes directly between cache and database:

• Pros: Resilient against cache outages; caches only queried data.
• Cons: Cache miss incurs 3 round trips (Cache → DB → Cache).

2. Read-Through

The application queries the cache layer exclusively. The cache transparently fetches missing records from the underlying DB and populates itself.

3. Write-Through

Data is written simultaneously to the Cache AND the Database before acknowledging success to the client:

• Pros: Strong data consistency; cache is never stale.
• Cons: High write latency (waits for both cache and disk write).

4. Write-Behind (Write-Back)

Data is written to the cache immediately, and asynchronously flushed to the database in background batches:

• Pros: Ultra-fast write throughput.
• Cons: Risk of permanent data loss if the cache node crashes before flushing to disk.

5. Write-Around

Data is written directly to the database, bypassing the cache entirely. The cache is only populated upon subsequent reads.

3. In-Memory Caches: Redis

Redis (Remote Dictionary Server) is the industry standard in-memory key-value data structure store.

Core Redis Data Structures:

Strings:      SET user:1001 '{"name":"Surya"}' EX 3600
Hashes:       HSET profile:1001 name "Surya" role "Admin"
Lists:        LPUSH user:1001:recent_posts "post_789"
Sets:         SADD article:12:tags "system-design" "redis"
Sorted Sets:  ZADD leaderboard 4500 "player_1" 9200 "player_2"
Bitmaps:      SETBIT active_users:2026-08-16 1001 1
HyperLogLog:  PFADD site_visitors "ip_192.168.1.1"

Production Example — Cache-Aside in Node.js / TypeScript:

import Redis from "ioredis";
const redis = new Redis();

async function getUserProfile(userId: string) {
  const cacheKey = `user:${userId}`;

  // 1. Check Redis Cache
  const cachedData = await redis.get(cacheKey);
  if (cachedData) {
    return JSON.parse(cachedData); // Cache HIT
  }

  // 2. Cache MISS — Query Primary DB
  const user = await db.query("SELECT * FROM users WHERE id = $1", [userId]);
  if (!user) return null;

  // 3. Populate Redis with 1 Hour TTL + Jitter to avoid stampede
  const ttl = 3600 + Math.floor(Math.random() * 300);
  await redis.setex(cacheKey, ttl, JSON.stringify(user));

  return user;
}

4. Cache Eviction Policies

When available cache memory is exhausted, eviction policies determine which keys are removed:

5. Classic Caching Pitfalls & Solutions

1. Cache Avalanche

• Problem: Millions of cache keys are set with the exact same TTL (e.g., midnight) and expire simultaneously, overwhelming the database with queries.
• Solution: Add random jitter to expiration times (TTL = 3600 + rand(0, 300)).

2. Cache Penetration

• Problem: Clients repeatedly query non-existent keys (e.g., user_id = -999), bypassing the cache and hitting the DB every time.
• Solution:
  1. Cache null values with short TTL (SET user:-999 "null" EX 60).
  2. Implement a Bloom Filter at the gateway to quickly reject non-existent IDs.

3. Cache Stampede (Thundering Herd)

• Problem: A heavily accessed key (e.g., top news article) expires, causing thousands of concurrent requests to all miss and query the DB at once.
• Solution: Use distributed locking (Redlock) or mutex locks so only one worker queries the DB while others wait.