Skip to content

Consumer apps at scale · 5 of 13

Step 1 · The pattern

Warm environments, fast answers

Steady traffic: 3 requests a second, each taking 150 ms. Two warm environments are enough: every request finds one idle and runs straight away. (All numbers are illustrative, and time is compressed.)

  • request
  • warm: fast

In short

Serverless cold starts: how it works and how it breaks

A serverless function runs in execution environments that the platform creates when they are needed. A request that finds a warm, idle environment runs straight away. Otherwise the platform first creates one: it starts the runtime, loads your code and runs its initialisation, which adds from under a tenth of a second to more than a second, longer for big runtimes and heavy start-up code. Each request in flight occupies one environment, so a spike after a quiet spell needs many new ones at once, and every early request pays the start-up cost. When the function reaches its concurrency limit, the platform throttles further requests with HTTP 429. For traffic you can predict, pre-initialise environments for the peak (provisioned concurrency on AWS Lambda), keep start-up code lean, and set the concurrency limit with room for the spike.

What this simulation simplifies

  • A cold start takes a fixed 1.5 s here; real start-up varies with the runtime, the code size and what the initialisation does.
  • Idle environments never expire within a step; real platforms reclaim idle environments after a while and recycle long-lived ones every few hours.
  • The limit is 10 environments for this one function; real limits are shared across functions in an account and Region.

How this comes up in interviews

"Your team is moving an API to serverless functions. Traffic is low most of the day with sharp spikes after push notifications. What do you watch out for?"

Then they ask

  1. Why are the first requests after a quiet period slow?
  2. What happens when a spike exceeds the function's concurrency?
  3. How would you prepare for a spike you can predict?

A strong answer covers

  • Cold starts: a request with no idle environment waits for the runtime and initialisation code to start.
  • Concurrency = requests in flight; each needs its own environment, so spikes need many new ones at once.
  • At the concurrency limit (or above the scaling rate) requests are throttled with 429; clients need back-off.
  • Provision environments ahead of predictable peaks, keep initialisation lean, and reserve concurrency so other functions cannot starve this one.
  • Watch downstream limits too: a function that scales fast can exhaust a database's connections.

Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.

On one page: the track cheat sheet.

Last reviewed: Official documentation

Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.