Skip to content

System design on a whiteboard

Status: AI draft, 2026-11-16. Not yet verified by me. Generic reference designs built from the linked pages; no real system is described.

In one line

Design out loud in a fixed order: requirements, the request path, the delivery path, where state lives, failure modes, security, and cost. Draw boxes and arrows as you go, state each trade-off as a choice with a reason, and leave room for "what if traffic grows ten times?"

The method

Step Questions to ask or answer
1. Requirements Users and traffic (requests per second, peaks), latency target, availability target, data size, regions, compliance, team size
2. Request path DNS, CDN, load balancer, compute, cache, database: draw it left to right
3. Delivery path How code gets from a commit to production, and how it rolls back
4. State Where data lives, how it is backed up, how it fails over, what is cached
5. Failure modes What happens when a zone, a node, a dependency or a deploy fails
6. Security Identity, network boundaries, secrets, encryption, audit
7. Cost The biggest line items and how they scale
8. Operations Metrics, logs, traces, alerts, runbooks

Spend the first few minutes on requirements: a design is only right for its requirements. Write the numbers on the board so later choices can point at them (High availability).

Design 1: a web application on EKS

flowchart TD
    U["Users"] --> R53["Route 53"]
    R53 --> CF["CloudFront + WAF"]
    CF --> ALB["ALB (public subnets,<br/>three zones)"]
    ALB --> PODS["App pods on EKS<br/>(private subnets)"]
    PODS --> REDIS["ElastiCache<br/>(cache)"]
    PODS --> DB["Aurora PostgreSQL<br/>(writer + readers)"]
    PODS --> S3["S3 (uploads)"]
  • Request path: Route 53 → CloudFront (static assets, TLS, WAF) → ALB created by the AWS Load Balancer Controller → pods spread across three zones (AWS Load Balancer Controller).
  • Compute: Deployments with HPA on CPU or requests; Karpenter adds nodes for pending pods; PDBs and topology spread keep zones balanced (Autoscaling, Karpenter in depth).
  • State: Aurora with readers in other zones; ElastiCache for hot reads with cache-aside; S3 for files (RDS and Aurora, Redis and caching patterns).
  • Security: pods use Pod Identity or IRSA for AWS access; secrets from Secrets Manager; security groups allow only the path shown (Cluster access, IRSA and Pod Identity).
  • Failure modes: a zone loss leaves two thirds of pods and a reader to promote; a bad deploy is caught by readiness and canary steps; a cache failure falls back to the database (check it can take the load).
  • Trade-offs to say: EKS gives control and portability but costs operational effort; ECS or serverless would be simpler for a small team (ECS and Fargate).

Design 2: a serverless API

flowchart TD
    C["Clients"] --> APIGW["API Gateway<br/>(auth, throttling)"]
    APIGW --> L["Lambda functions"]
    L --> DDB["Database<br/>(DynamoDB or Aurora via RDS Proxy)"]
    L --> Q["SQS queue"]
    Q --> W["Worker Lambda"]
    W --> S3["S3"]
  • Request path: API Gateway handles TLS, authorization (Cognito or JWT authorizer) and throttling; Lambda runs the handlers (API Gateway, Cognito).
  • Asynchronous work: slow tasks go to SQS and a worker function with a dead-letter queue (SQS, SNS and EventBridge).
  • State: DynamoDB fits key-value access at any scale; a relational database needs RDS Proxy to survive Lambda's connection fan-out.
  • Limits to mention: API Gateway integration timeouts, Lambda's maximum duration and concurrency, payload sizes (Lambda).
  • Trade-offs to say: no servers to patch and cost scales to zero; cold starts, per-request cost at high steady traffic, and harder local testing.

Design 3: a CI/CD pipeline with GitOps

flowchart TD
    DEV["Developer push"] --> CI["CI: test, scan, build"]
    CI -->|"push image by digest"| ECR["ECR"]
    CI -->|"PR: bump image tag"| GITOPS["GitOps repo"]
    GITOPS --> ARGO["Argo CD in each cluster"]
    ARGO --> STG["Staging"]
    ARGO --> PROD["Production<br/>(after approval)"]
  • CI: tests, linting, dependency and image scanning, build once, push by digest, sign; CI authenticates to AWS with OIDC, not stored keys (OIDC from pipelines to AWS, Pipeline security).
  • CD: the pipeline changes Git, not the cluster; Argo CD pulls and syncs; promotion is a pull request from staging to production values (Principles, Promotion and rollback).
  • Safety: required reviews, protected branches, canary or blue/green in production, automated rollback on error-rate alerts (Deployment strategies).
  • Trade-offs to say: GitOps gives an audit trail and drift correction, but adds a repository and a controller to run; a push-based pipeline is simpler for one cluster.

Questions interviewers add, and how to handle them

Follow-up What to talk about
"Traffic grows ten times" Which component saturates first: database connections and writes, then cache, then compute; read replicas, caching, queues, sharding
"Make it multi-region" Active-passive versus active-active, data replication lag, DNS failover, cost doubling (Backup and DR)
"Cut the cost in half" Right-sizing, Spot for stateless pods, Savings Plans, storage tiers, NAT and data transfer (Cost optimisation)
"How do you know it works?" SLOs, golden-signal dashboards, alerts on symptoms, synthetic checks (SLOs)
"A whole zone fails" Spread across three zones, health checks, failover of the database, capacity headroom

Habits

  • Say numbers out loud and use them: "1,000 requests per second at 100 KB each is about 100 MB per second."
  • Name the trade-off for every box: why this and not the obvious alternative.
  • Keep it simple first, then evolve when asked; a design with fewer moving parts is easier to defend.
  • Admit what you would look up: exact quotas and prices change (How to answer technical questions).