2 min read messaging-queue

Kafka vs RabbitMQ

Kafka vs RabbitMQ

This is the second part of the already published Part I which can be found below

Kafka vs RabbitMQ
Are RabbitMQ and Kafka same ?
The BitmaskersTirthankar Kundu

Now let's dive a bit deep into the differences between Kafka and RabbitMQ.

The Architecture

RabbitMQ

  • General purpose message broker— Supports request/reply, point-to-point, and pub-sub communication patterns.
  • Smart broker / dumb consumer model— Consistent delivery of messages to consumers.
  • Mature platform— Available for Java, .NET, Ruby, Node, etc. Also offers many plugins.
  • Communication— Can be synchronous or asynchronous.
  • Deployment scenarios— Provides distributed deployment scenarios.
  • Multi-node cluster to cluster federation— Can form clusters using declaratively or manually.

Apache Kafka

  • High volume publish-subscribe messages and streams platform—Durable, fast, and scalable.
  • Durable message store— Run in a server cluster, which keeps streams of records in topics.
  • Messages— Made up of a value, a key, and a timestamp.
  • Dumb broker / smart consumer model— Does not try to track which messages are read by consumers and only keeps unread messages. Kafka keeps all messages for a set period of time.
  • Requires external services to run— Apache Zookeeper.
The Approach

RabbitMQ: Push-based approach

RabbitMQ uses a push model where data is sent from the Broker to the Consumer. This can be used for low-latency messaging.

The push model's goal is to distribute messages individually and efficiently, ensuring that work is parallelized equally and messages are handled roughly in the order they arrived in the queue.

Apache Kafka: Pull-based approach

Kafka uses a pull model. Consumers request batches of messages from a specific offset. Pull-based systems have some deficiencies like resource wasting due to polling regularly. Kafka supports a 'long polling' waiting mode until real data comes through to alleviate this drawback.

Because of Kafka's partitions, a pull model makes sense. In a partition with no competing users, Kafka provides message order. This helps users to take advantage of message batching for more efficient message delivery and higher throughput.

Message Handling

RabbitMQ

  • Doesn't support Message Ordering
  • Messages are not retained after consumption since it's a queue
  • Supports Message priorities

Apache Kafka

  • Provides message ordering because of its partition
  • Messages are always there and can be managed using a message retention policy
  • Doesn't support Message priorities
The Performance

Apache Kafka:

Compared to message brokers like RabbitMQ, Kafka provides significantly better efficiency. It improves performance by using sequential disc I/O, making it a good option for queue implementation. With limited resources, it can achieve high throughput (millions of messages per second), which is important for big data use cases.

RabbitMQ:

RabbitMQ can also handle a million messages per second, but it does so at the expense of more resources(around 30 nodes). RabbitMQ can be used for many of the same applications as Kafka, but it must be used in combination with other software such as Apache Cassandra.

Tirthankar Kundu