Design a rate limiter

Question 1

Who is being limited?

Accepted Answer

The caller is the thing you are counting. Put it on the board.

Question 2

Where is the decision actually made?

Accepted Answer

A limiter has to sit where every request passes and where rejecting one is cheap — the gateway, or middleware in front of the handler. Behind the expensive work is too late to save anything.

Question 3

Forty servers each hold their own counter. What is the real limit?

Accepted Answer

Forty times whatever you configured. Counters kept per-server are not a shared limit at all. The count has to live somewhere every server can read and increment atomically — which in practice means an in-memory store like Redis, close enough that the hop does not dominate the request.

Question 4

What are you protecting?

Accepted Answer

The limiter exists to keep something else standing. That something should be on the board.

Question 5

Can the thing making the decision actually read the counter?

Accepted Answer

The enforcement point and the shared counter have to be connected, or the decision is being made on data the server does not have.

Question 6

Which counting algorithm, and what does it cost you?

Accepted Answer

Fixed windows are trivial and let through up to twice the limit around a boundary. Sliding windows and token buckets fix that and cost more to store and update. Naming the trade rather than the algorithm is the answer an interviewer is listening for.

Blueprint — system design