Stage 4 · Redundancy · Failover · Single Point of Failure
Always On
Your payments service can't blink. At peak, a hardware fault takes the primary database offline for 20 seconds. Adding more capacity won't save you — a dead database at any size is still dead. Design so a single component failure never takes the whole system down.
Requirements
Functional
·Payments service keeps serving requests through a primary database outage
Non-functional
·~100 requests/sec
·50/50 read-write split
·availability ≥ 95% through a 20-second primary outage
Constraints
·Budget: $12/hr or less
·Every database must sit behind the API — never exposed to the client
·More capacity does not fix an outage — only redundancy does
Out of scope
·Multi-region disaster recovery
TrafficRequests per second — the rate of incoming traffic this level throws at your system.
~100 r/s · mixed
50% reads · 50% writesWhat fraction of requests are reads (fetches) vs writes (updates) — this decides which components sit on the hot path.
Win conditions
p99 latency99% of requests finish at or under this time — a stricter tail-latency bar than p95 that catches worst-case slowness.≤ 400ms
availabilityThe share of legitimate requests that succeeded, out of everything sent your way. 99% still means 1 in 100 users saw an error.≥ 95%
throughputHow many requests per second your system actually completed — not requested, completed.≥ 45 r/s
error rateThe share of requests that failed outright: timeouts, drops, or capacity overload.≤ 5%
costWhat your architecture costs to run per hour, based on the instances and components you've wired up.≤ $12/hr
During the run
sql db outageat 20s · 20s
Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.
Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.
Drag components from the right → connect their handles → Run.
Drag onto canvas
Components
Client
ClientWhere traffic originates.
Compute
API GatewayFront door — the app tier requests pass through.
500 r/s · 20ms · $2/hr
BackendA second compute tier for heavier processing.
350 r/s · 25ms · $2/hr
Storage
SQL DatabaseRelational store. Durable, but the lowest throughput.
200 r/s · 50ms · $3/hr
NoSQL DB3× the throughput of SQL — at a premium. Pay for it only when the load demands it.
600 r/s · 30ms · $4/hr
Redis CacheIn-memory read-cache — absorbs repeated reads. Useless for writes.
5000 r/s · 3ms · $2/hr
Read ReplicaRead-only copy of the primary — scales reads past the primary's own ceiling.
200 r/s · 55ms · $3/hr
DB StandbyHot backup that takes over only when the primary fails.
200 r/s · 50ms · $3/hr
Networking
CDNEdge read-cache — serves content near the user.
50000 r/s · 5ms · $1/hr
Load BalancerManaged, premium front door — highest ceiling, lowest latency, costs more.
10000 r/s · 2ms · $2/hr
Reverse ProxyCheaper self-hosted front door — lower ceiling, the smart buy under 8,000 r/s.
8000 r/s · 3ms · $1/hr
Messaging
QueueACKs writes instantly, drains them to storage — absorbs bursts, not sustained overload.