System Design — Complete Notes

System Design is the process of defining the architecture, components, modules, interfaces, and data flow of a system to satisfy specified requirements. It focuses on making software that is scalable, reliable, available, and maintainable.

Example: Designing Instagram at Scale

Questions System Design Answers

• How do we support 10 million concurrent users?
• How do we make the application fast (< 200ms response)?
• What happens if a server crashes?
• Where do we store millions of images/videos?
• How do we prevent the database from becoming a bottleneck?
• How do we scale from 1,000 req/s to 100,000 req/s?

1. Networking Fundamentals

Understanding networking is the foundation of system design. Every system communicates over a network.

1.1 What Happens When You Type a URL?

This is one of the most important concepts in system design interviews.

Step-by-step breakdown:

1. Browser parses the URL → Protocol (https), domain (google.com), path (/search)
2. DNS Lookup → Resolves google.com to an IP address like 142.250.183.14
3. TCP Connection → Three-way handshake (SYN → SYN-ACK → ACK)
4. TLS Handshake → If HTTPS, negotiate encryption (Client Hello → Server Hello → Key Exchange)
5. HTTP Request → Browser sends GET /search?q=hello HTTP/2
6. Server Processing → Server receives request, queries database, builds response
7. HTTP Response → Server returns 200 OK with HTML content
8. Rendering → Browser parses HTML, fetches CSS/JS, paints the page

1.2 HTTP / HTTPS

HTTP (HyperText Transfer Protocol) is the foundation of data communication on the web.

HTTP Methods:

HTTP Status Codes:

1.3 TCP vs UDP

TCP Three-Way Handshake:

1.4 DNS (Domain Name System)

DNS is the "phonebook of the internet" — it translates human-readable domain names to IP addresses.

DNS Resolution Process:

DNS Record Types:

1.5 REST APIs

REST (Representational State Transfer) is an architectural style for designing networked applications.

REST Principles:

1. Client-Server — Separation of concerns
2. Stateless — Each request contains all needed information
3. Cacheable — Responses can be cached
4. Uniform Interface — Consistent URL structure
5. Layered System — Client doesn't know if it's talking to end server or intermediary

Example REST API:

GET    /api/users          → List all users
GET    /api/users/123      → Get user 123
POST   /api/users          → Create a new user
PUT    /api/users/123      → Replace user 123
PATCH  /api/users/123      → Update user 123 partially
DELETE /api/users/123      → Delete user 123

1.6 WebSockets

WebSockets provide full-duplex, persistent connections between client and server.

Real-world uses: Chat apps (Slack, Discord), live sports scores, stock tickers, multiplayer games.

1.7 Latency vs Bandwidth

Latency Numbers Every Programmer Should Know:

2. Client-Server Architecture

The client-server model is the backbone of virtually every web application.

2.1 How It Works

• Client: Makes requests (browser, mobile app, another server)
• Server: Processes requests, runs business logic, returns responses
• API: The contract between client and server

2.2 Stateless vs Stateful Servers

Best Practice: Design stateless servers and externalize state (Redis, database).

2.3 Vertical vs Horizontal Scaling

)

In practice: Start with vertical scaling, then move to horizontal when you hit limits.

2.4 Monolith vs Microservices

When to use what:

• Monolith: Startups, MVPs, small teams, simple domains
• Microservices: Large orgs, complex domains, need for independent deployments

3. Databases 🗄️

Databases are the heart of any system. Choosing the right database is one of the most critical design decisions.

3.1 SQL Databases (Relational)

SQL databases store data in tables with rows and columns and enforce relationships via foreign keys.

Popular SQL Databases:

ACID Properties:

Indexes:

Indexes speed up reads dramatically but slow down writes.

-- Without index: Full table scan O(n)
SELECT * FROM users WHERE email = 'john@example.com';

-- With index: B-tree lookup O(log n)
CREATE INDEX idx_users_email ON users(email);
SELECT * FROM users WHERE email = 'john@example.com';  -- 100x faster

When Index Helps:

3.2 NoSQL Databases

NoSQL databases are designed for specific data models and have flexible schemas.

Types of NoSQL Databases:

3.3 SQL vs NoSQL — When to Use What?

Decision Framework:

Need ACID transactions?          → SQL (PostgreSQL, MySQL)
Flexible/changing schema?        → NoSQL (MongoDB)
Ultra-fast key lookups?          → Key-Value (Redis, DynamoDB)
Massive write throughput?        → Column (Cassandra)
Complex relationships?           → Graph (Neo4j) or SQL
Time-series data?                → InfluxDB, TimescaleDB

4. Database Scaling

As your application grows, a single database becomes a bottleneck. Database scaling strategies are critical.

4.1 Replication

Replication creates copies of your database across multiple servers.

Primary-Replica Architecture:

4.2 Sharding (Horizontal Partitioning)

Sharding splits data across multiple databases based on a shard key.

Example: Sharding users by user_id

Shard Key: user_id % 3

User ID 1  → Shard 0
User ID 2  → Shard 1
User ID 3  → Shard 2
User ID 4  → Shard 0  (4 % 3 = 1... wraps around)
User ID 5  → Shard 1
User ID 6  → Shard 2

Sharding Strategies:

Challenges of Sharding:

• Cross-shard queries are expensive
• Rebalancing when adding new shards
• JOINs across shards are nearly impossible
• Referential integrity can't be enforced across shards

4.3 CAP Theorem

The CAP theorem states that a distributed system can guarantee only two out of three properties simultaneously.

In practice, network partitions are inevitable, so you're really choosing between:

4.4 Consistency Models

5. Caching ⚡

Caching stores frequently accessed data in fast storage (memory) to reduce latency and database load.
)

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

5.1 Why Caching Matters

5.2 Caching Strategies

Cache-Aside Pattern (Most Common):

5.3 Redis

Redis (Remote Dictionary Server) is the most popular in-memory data store used for caching.

Redis Data Structures:

Strings:   SET user:123 "John"            → Simple key-value
Hashes:    HSET user:123 name "John"      → Object-like storage
Lists:     LPUSH notifications "msg1"     → Queues, feeds
Sets:      SADD tags "python" "redis"     → Unique collections
Sorted Sets: ZADD leaderboard 100 "player1" → Rankings, priority queues

Redis Example — Cache-Aside in Node.js:

async function getUser(userId) {
  // 1. Check cache
  const cached = await redis.get(`user:${userId}`);
  if (cached) return JSON.parse(cached); // Cache HIT

  // 2. Cache MISS — query database
  const user = await db.query("SELECT * FROM users WHERE id = $1", [userId]);

  // 3. Store in cache with 1 hour TTL
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));

  return user;
}

5.4 Cache Eviction Policies

When the cache is full, which items do we remove?

5.5 Cache Problems

6. Load Balancers

A load balancer distributes incoming traffic across multiple servers to ensure no single server is overwhelmed.

6.1 Why Load Balancers?

Benefits:

• High Availability — If one server dies, traffic reroutes to healthy servers
• Scalability — Add/remove servers without downtime
• Performance — Distribute load evenly
• SSL Termination — Offload HTTPS encryption from app servers

6.2 Layer 4 vs Layer 7 Load Balancing

6.3 Load Balancing Algorithms

6.4 Health Checks

Load balancers continuously check server health:

Health check types:

• Active: LB periodically pings servers
• Passive: LB monitors response errors from real traffic

6.5 Reverse Proxy vs Load Balancer

Note: Nginx can function as both a reverse proxy and a load balancer.

7. Message Queues & Event-Driven Systems

Message queues enable asynchronous communication between services, decoupling producers from consumers.

7.1 Why Message Queues?

Without a queue (synchronous) vs With a queue (asynchronous):

7.2 Core Concepts

7.3 Apache Kafka

Kafka is a distributed event streaming platform designed for high-throughput, fault-tolerant messaging.

Kafka Architecture:

Why Kafka?

7.4 Pub/Sub vs Point-to-Point

7.5 Dead Letter Queues (DLQ)

When a message fails processing after max retries, it goes to a Dead Letter Queue for investigation.

8. Distributed Systems

A distributed system is a collection of independent computers that appear to the user as a single coherent system.

8.1 Why Distributed Systems?

8.2 Challenges (Fallacies of Distributed Computing)

Peter Deutsch's 8 Fallacies — things developers wrongly assume:

1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous

8.3 Consensus Algorithms

How do multiple nodes agree on a value?

Raft — Leader Election:

8.4 Distributed Locks

When multiple services need exclusive access to a resource:

Redis-based Distributed Lock (Redlock):

1. Client tries to SET lock_key "unique_value" NX EX 30
   NX = only if not exists
   EX = expires in 30 seconds

2. If SET succeeds → lock acquired ✅
3. Do work...
4. Release: DELETE lock_key (only if value matches)

8.5 Idempotency

An operation is idempotent if performing it multiple times has the same effect as performing it once.

Why it matters: In distributed systems, requests can be retried due to network failures. Idempotent operations are safe to retry.

9. Storage

Understanding storage types is crucial for designing systems that handle large amounts of data.

9.1 Storage Types

9.2 Object Storage (S3)

Amazon S3 is the gold standard for object storage.

S3 Key Concepts:

Bucket: my-app-images
  ├── users/avatar/123.jpg     (Object)
  ├── users/avatar/456.jpg     (Object)
  ├── posts/images/789.png     (Object)
  └── backups/db-2024-01.sql   (Object)

Each object has:
  - Key:      users/avatar/123.jpg
  - Value:    Binary data (the file)
  - Metadata: Content-Type, size, custom headers

S3 Storage Classes:

9.3 When to Use What?

Structured data + queries?     → Database (PostgreSQL, MongoDB)
User-uploaded files?           → Object Storage (S3)
Shared file system?            → File Storage (EFS, NFS)
VM/Database disk?              → Block Storage (EBS)
Static website assets?         → Object Storage + CDN
Large media files?             → Object Storage with CDN

10. CDN (Content Delivery Network) 🌍

A CDN is a geographically distributed network of servers that delivers content to users from the nearest edge location, reducing latency.

10.1 How CDN Works

10.2 Push vs Pull CDN

10.3 CDN Use Cases

Popular CDNs: Cloudflare, AWS CloudFront, Akamai, Fastly, Google Cloud CDN

10.4 Cache Invalidation in CDN

How to update content when it changes:

11. Microservices

Microservices architecture structures an application as a collection of small, autonomous services that are independently deployable.

11.1 API Gateway

An API Gateway is the single entry point for all client requests in a microservices architecture.

11.2 Service Communication

Synchronous vs Asynchronous Communication:

11.3 Service Discovery

How do services find each other?

11.4 Circuit Breaker Pattern

Prevents cascading failures when a downstream service is failing.

Three States:

11.5 Saga Pattern (Distributed Transactions)

Since microservices can't use traditional database transactions across services, use the Saga pattern:

Example: E-commerce Order

12. Reliability & Fault Tolerance

Building systems that continue operating correctly even when components fail.

12.1 Key Concepts

Availability SLAs (The "Nines"):

12.2 Redundancy & Failover

12.3 Retry Strategies

Exponential Backoff with Jitter:

Attempt 1: wait 1s + random(0, 0.5s)
Attempt 2: wait 2s + random(0, 1.0s)
Attempt 3: wait 4s + random(0, 2.0s)
Attempt 4: wait 8s + random(0, 4.0s)
Max retries: 5 → give up, alert

12.4 Rate Limiting

Controls how many requests a client can make in a time window.

Common Algorithms:

12.5 Graceful Degradation

When part of the system fails, reduce functionality instead of complete failure:

Full Service:
  - Real-time recommendations (ML model)
  - Live search suggestions
  - Dynamic pricing

Degraded Service (ML model down):
  - Static popular recommendations ← fallback
  - Basic search (no suggestions)
  - Standard pricing ← fallback

13. Security 🔐

Security is a non-negotiable aspect of system design.

13.1 Authentication vs Authorization

13.2 OAuth 2.0

OAuth 2.0 is the industry standard for authorization. It lets apps access user data without exposing passwords.

Authorization Code Flow (Most Common):

OAuth Roles:

13.3 JWT (JSON Web Tokens)

JWTs are self-contained tokens used for stateless authentication.

JWT Structure:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.     ← Header (base64)
eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikp.   ← Payload (base64)
SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQ.    ← Signature

Header:  { "alg": "HS256", "typ": "JWT" }
Payload: { "sub": "1234567890", "name": "John", "role": "admin", "exp": 1700000000 }
Signature: HMACSHA256(base64(header) + "." + base64(payload), secret)

JWT vs Session:

13.4 HTTPS & TLS

TLS (Transport Layer Security) encrypts communication between client and server.

TLS Handshake:

13.5 Common Security Threats

Password Security:

❌ Never store plain text: "password123"
❌ Never use MD5/SHA1:     "482c811da5d5b4bc..."
✅ Use bcrypt/scrypt:      "$2b$12$LJ3m4ys..." (salted + slow hash)

13.6 API Security Checklist

1. ✅ Use HTTPS everywhere
2. ✅ Authenticate with OAuth 2.0 / JWT
3. ✅ Implement rate limiting
4. ✅ Validate and sanitize all inputs
5. ✅ Use parameterized database queries
6. ✅ Set proper CORS headers
7. ✅ Don't expose sensitive data in URLs
8. ✅ Log security events
9. ✅ Keep dependencies updated
10. ✅ Use API keys for server-to-server communication

14. Observability 🔍

Observability is the ability to understand the internal state of a system by examining its outputs.

14.1 Three Pillars of Observability

14.2 Logging

Log Levels:

Structured Logging (Best Practice):

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "ERROR",
  "service": "payment-service",
  "trace_id": "abc-123-def",
  "user_id": "user_456",
  "message": "Payment processing failed",
  "error": "InsufficientFunds",
  "amount": 99.99,
  "currency": "USD"
}

14.3 Metrics

Key Metrics to Monitor:

These are known as the Four Golden Signals (from Google's SRE book).

Prometheus Query Examples:

# Request rate (last 5 min)
rate(http_requests_total[5m])

# 99th percentile latency
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))

# Error rate
rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m])

# CPU usage
100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

14.4 Distributed Tracing

In microservices, a single user request can span multiple services. Tracing shows the complete journey.

Example Trace:

Each span contains:

• Trace ID (groups all spans)
• Span ID (unique to this operation)
• Parent Span ID
• Start time, duration
• Tags and logs

14.5 Monitoring & Alerting Stack

Typical Production Stack:

Tools Worth Knowing:

14.6 Alerting Best Practices

🧭 Quick Reference — System Design Building Blocks