Design a notification service

Question 1

What do the calling services actually call?

Accepted Answer

One API in front of every channel is the whole point of this service: the caller says "notify this user", not "open an SMTP connection". Without it every team writes its own retry logic against a different provider.

Question 2

The SMS provider is timing out. What happens to the request that arrived a moment ago?

Accepted Answer

Sending inside the request means the caller waits on somebody else’s outage, and a failed send is simply lost. Accept, enqueue, return — then the worker can retry with backoff for an hour and nobody upstream notices.

Question 3

What actually talks to the providers?

Accepted Answer

Each channel has its own provider, its own rate limits and its own failure modes. Workers pulling from the queue let you retry SMS without holding up push, and add capacity to one channel alone.

Question 4

Who delivers the actual message to the phone?

Accepted Answer

You do not run the App Store’s push network or the phone network. Those are third parties on the board, and drawing them is what makes it obvious that most of your failure modes are theirs.

Question 5

A customer says they never got the email. How do you answer?

Accepted Answer

Every send needs a durable record — what was sent, to whom, over which channel, and what the provider said. It is also what stops a retry after a crash turning into a second notification.

Question 6

You look up preferences and templates on every one of a thousand sends a second. Where do they come from?

Accepted Answer

Preferences and templates are tiny, read constantly and change rarely — the easiest cache in the system. Hitting the database for them on every send turns a 1,000/s service into 1,000 queries a second for data that barely moves.

Question 7

How many notifications a second is 100M a day?

Accepted Answer

100,000,000 / 86,400 is about 1,160 a second. Worth doing early, because it tells you the interesting problem here is provider failure and de-duplication, not throughput.

Blueprint — system design