RabbitMQ vs Kafka vs Redis: Choosing a Message Queue
These three get grouped together but they are different tools: a task queue, an event log, and a lightweight middle ground. Here is what each is for, what it costs to run, and how to choose.
RabbitMQ, Kafka and Redis all move messages between parts of a system, which is why they land in the same conversation. They are built on different models: RabbitMQ is a task queue with rich routing, Kafka is a persistent event log you can replay, and Redis is an in-memory store that can do lightweight queuing. Choosing well means matching the model to what you actually need.
RabbitMQ vs Kafka vs Redis at a glance
| Aspect | RabbitMQ | Kafka | Redis (Streams / lists) |
|---|---|---|---|
| Model | Task queue, message broker | Distributed append-only log | In-memory store, simple queues |
| Delivery | At-least-once, acks per message | At-least-once, consumer tracks offset | At-least-once with Streams, at-most-once with lists |
| Ordering | Per queue | Strict per partition | Per stream |
| Retention / replay | Consumed then gone | Kept for a window, fully replayable | Limited, bounded by memory |
| Throughput | High | Very high | Very high, bounded by memory |
| Ops complexity | Moderate | High (brokers, ZooKeeper/KRaft, partitions) | Low, often already in your stack |
| Typical use | Background jobs, RPC, work distribution | Event streaming, event sourcing, analytics pipelines | Job queues, simple pub/sub, rate limiting |
RabbitMQ: a real message broker
RabbitMQ shines when you need routing: exchanges and bindings let you fan a message out to several queues, route by a key, or filter by headers, and consumers acknowledge each message so redelivery is precise. It is the right choice for distributing background work across worker pools, for request-reply patterns, and for any workflow where the routing logic is not trivial.
Kafka: a log you can replay
Kafka is not a queue; it is an ordered, persistent log that many consumers read independently, each tracking its own position. Because messages stay for a retention window, a new consumer can replay history, and you can rebuild a downstream store from the log. That makes it the tool for event streaming, event sourcing, and feeding several analytics or search pipelines from one source of truth. The cost is operational: partitions, replication, consumer group rebalancing, and a cluster to run.
Redis: the lightweight middle ground
Redis is probably already in your stack for caching, and Redis Streams or plain lists handle a job queue well enough for a lot of applications. Laravel's queue driver on Redis, with Horizon for visibility, is a common and solid setup. The limits are real: everything lives in memory, retention is bounded, and the routing and replay features of the other two are not there. Use it until you have a concrete reason not to.
Publishing reliably: the outbox pattern
Whichever broker you pick, publishing an event in the same breath as a database write is not atomic: the write can commit and the publish can fail, or the reverse. Write the event to an outbox table inside the same transaction as the business change, then a separate process reads the outbox and publishes to the broker. This is how you get exactly-one publish without distributed transactions.
With Laravel
For background jobs, the built-in queue on Redis plus Horizon covers most needs with almost no setup. For RabbitMQ, a community queue driver connects the same Job classes to a RabbitMQ backend. For Kafka, packages exist but you are usually integrating Kafka for event streaming across services rather than as a Laravel job queue, so you produce and consume it directly.
A decision guide
- ✓Background jobs, a handful of services, no replay need: Redis queue plus Horizon.
- ✓Complex routing, work distribution across worker pools, request-reply: RabbitMQ.
- ✓Event streaming, event sourcing, replay, many consumers off one source: Kafka, and budget for the operations.
- ✓Not sure and on Laravel: start with Redis. Moving to RabbitMQ later is a driver change; moving to Kafka is an architecture change you make deliberately.
FAQ
- RabbitMQ or Kafka?
- RabbitMQ if you need rich routing and precise per-message acknowledgement for distributing background work. Kafka if you need a replayable, ordered log that several consumers read independently, for event streaming or event sourcing. They are not interchangeable: one is a broker, the other is a log.
- Is Redis enough as a message queue?
- For background jobs in an application with a handful of services and no need to replay history, yes. Redis Streams and Laravel's Redis queue with Horizon are a common, reliable setup. You outgrow it when you need routing, durable replay, or throughput beyond what memory allows.
- Is Kafka overkill for my project?
- Often, yes. Kafka earns its operational cost when you have event streaming across many services, need to replay events to rebuild state, or feed several analytics and search pipelines from one log. For background jobs in a single application, it is more machinery than the problem requires.
- What is the difference between a task queue and an event log?
- A task queue delivers each message to one consumer, which processes it and acknowledges it, after which it is gone. An event log keeps messages for a retention window and lets many consumers read them independently at their own pace, and replay them. RabbitMQ is the first; Kafka is the second.
- How do I make sure a message is not lost?
- Use at-least-once delivery with consumer acknowledgements, make consumers idempotent so a redelivered message is safe, and publish events with the outbox pattern so the publish cannot silently fail after the database write commits. Then monitor consumer lag and dead-letter queues.
RabbitMQ, Kafka and Redis solve three related but distinct problems: distributing work, streaming replayable events, and lightweight queuing. On Laravel, start with the Redis queue, move to RabbitMQ when routing gets real, and adopt Kafka only when event streaming is genuinely part of your architecture.
Need help with this topic? Microservices Migration
Discover this service →