Skip to content
Build
Break
Build
Break
Learn system design by watching real systems break.
Today
Outages
Lab
Practice
Learn
Library
Search
Ctrl K
All tracks
Track · 0/6 done
LLM inference serving
Serve a model to many users at once: batches, GPU memory, queues, streaming tokens and gateways.
01
Lesson
· 2 min
Batching vs latency
02
Lesson
· 2 min
KV cache memory limits
03
Lesson
· 2 min
GPU queueing
04
Lesson
· 2 min
Token streaming
05
Lesson
· 2 min
LLM gateway
06
Read
· HTTP reference · 429
429 Too Many Requests
07
Lesson
· 2 min
Long-generation timeouts
08
Read
· HTTP reference · 504
504 Gateway Timeout
Revise
LLM inference serving cheat sheet
Key concepts, decision rules and traps on one printable page
Today
Outages
Lab
Practice
Learn
Library
BuildBreak needs JavaScript for its simulations.