Design a durable message queue in the shape of Kafka: producers append to a topic, several independent consumer groups read it, and nothing is lost when a broker restarts.
Requirements
- Append a message to a topic and get an acknowledgement
- Several consumer groups read the same topic independently
- A consumer resumes where it left off after a crash
- Messages survive a broker dying
Scale
- Messages
- 1M per second
- Message size
- 1KB
- Retention
- 7 days
- Consumers
- several groups per topic
Draw the design on the board, fill in Your answer, then submit.