Message Queue
Asynchronous communication buffer between services. Decouples producers from consumers and provides durability during traffic spikes.
What Is a Message Queue?
A message queue is a durable buffer that holds messages from producers until consumers process them. The producer and consumer are fully decoupled: neither knows about the other's availability or speed.
Core Benefits
- Decoupling: Services fail and recover independently.
- Load leveling: Absorbs traffic spikes. Consumers process at their own pace.
- Durability: Messages persist until acknowledged, preventing data loss.
Kafka vs SQS vs RabbitMQ
| Kafka | SQS | RabbitMQ | |
|---|---|---|---|
| Model | Log-based (pull) | Queue (poll) | Queue (push/pull) |
| Retention | Time-based (7d default) | Until consumed | Until consumed |
| Ordering | Per-partition | Best-effort (FIFO queue option) | Per-queue |
| Throughput | Very high | High | Medium |
| Use case | Event streaming, replayability | Task queues, AWS-native | Complex routing, RPC |
Interview Pattern
Whenever a design involves an async operation (email sending, image processing, notifications), propose a message queue. Mention at-least-once delivery and idempotent consumers.
Related Concepts
Controls the rate of requests to a service. Common algorithms: Token Bucket, Leaky Bucket, Fixed Window, and Sliding Window.
Horizontal partitioning of a database across multiple machines to distribute load beyond a single server's capacity.