The idea: delete on ack, or keep and move an offset
A queue and a log both sit between producers and consumers, but they store messages in opposite ways. A RabbitMQ queue hands each message to one consumer, waits for its ack, and then deletes it. A Kafka log appends each record at the next offset and keeps it. A consumer does not remove anything: its consumer group just remembers how far it has read. The page sends the same 8 order events, keyed by customer (A1 A2 B1 B2 A3 B3 A4 B4), into both and lets you consume them.
Delete vs keep
In RabbitMQ the broker sends a message with basic.deliver. Until the consumer sends basic.ack the message is unacked (yellow on the canvas): still in the queue, but not given to anyone else. On the ack the broker deletes it. In Kafka the consumer calls poll(), handles the records and commits the next offset (commitSync). The record stays where it is. Demo: one service, delete vs keep ends with an empty queue (4 deleted on ack) and a log that still holds all 4 records, with billing at offset 2 in both partitions.
Replay and late joiners
Because the log keeps records, a group can seek back to the start and read them again, for example after fixing a bug (Demo: replay: billing reads A1 and B1 a second time). A queue cannot: the acked messages are gone and the producer would have to send them again.
The same goes for a new service. In Demo: fan-out to a late second service, analytics is added after 4 events. Its new RabbitMQ queue is bound to the fanout exchange and gets only what is published later: A3 and B3. Its new Kafka group starts at the earliest offset (auto.offset.reset = earliest) and can read all 6 records; after 4 rounds it has read 4 and its lag is 2.
Competing consumers and ordering
To go faster, a service adds workers. On a queue they are competing consumers: all of them read the same queue and the broker hands messages out round-robin. Any worker can get any message. In Demo: competing consumers and ordering, billing-1 (slow) gets A1 and billing-2 gets A2; billing-2 finishes first, so A2 is handled before A1. For order events of one customer that is a bug.
A Kafka topic is split into partitions, and the producer picks the partition from the record's key (here A → p0, B → p1). Inside a group each partition has exactly one owner, so all of customer A's records go to one worker, in offset order. The same demo keeps the order per key ✓. The price is that a group can use at most as many workers as there are partitions: in Demo: a third worker billing-3 gets no partition and is idle. The queue spreads 6 messages over 3 workers in 2 rounds; the log handles 4 in the same time.
Fan-out
When several services want every message, RabbitMQ binds one queue per service to a fanout exchange, which copies each message into each queue. Kafka writes each record once; every service is a separate group with its own offsets on the same log. In the fan-out demo the queue side writes 8 copies, the log side 6 records.
Retention
A queue keeps an unread message until someone consumes it (unless a TTL or a length limit is set). A slow consumer makes the queue grow. A log deletes old segments when retention.ms (7 days by default) or retention.bytes is reached, read or not. A group that is further behind than that silently skips ahead. In Demo: retention billing has read A1 and B1 when the oldest records expire, so it never sees A2 and B2 (missed 2 ✗), while the queue still has its 6 unread messages.
When to pick which
| Queue (RabbitMQ) | Log (Kafka) | |
|---|---|---|
| After a message is handled | Deleted on ack | Kept until retention; the group's offset moves |
| Replay, new service reads history | No | Yes: seek, or start at the earliest offset |
| Scaling workers | Any number on one queue | At most one per partition in a group |
| Order per key with many workers | Not kept ✗ | Kept within a partition ✓ |
| Fan-out to N services | N queues, N copies | One log, N groups |
| Slow consumer | Queue grows, nothing lost | Lag grows; past retention, records are missed |
| Good fit | Task queues, per-message routing, retries, dead-lettering | Event streams, many readers, reprocessing, ordered per key |
See also Push vs Pull Messaging for how the two deliver (prefetch vs fetch), How Kafka Moves a Message for the full Kafka path, Kafka Consumer Groups for how partitions are assigned, and Delivery Semantics for what happens when a worker crashes between handling and ack or commit.
What the page leaves out
Nobody crashes, so there are no redeliveries, requeues or rebalances after a failure. One worker handles one message per round, with prefetch_count = 1 and one record per poll. It leaves out RabbitMQ's single active consumer and consistent-hash exchange (which give per-key order on queues), quorum queues, streams (RabbitMQ's own append-only log), TTLs, dead-letter exchanges and nack, and Kafka's replication, log compaction, segment files, consumer lag alerts and transactions. A "round" is a teaching clock, not real time.