In an event-driven architecture, event routers are the components that connect event consumers to event producers. Not all implementations of event routers are the same, nor do any of them offer an all-purpose solution, so deciding which one to use depends on your use case and project's needs.
Understanding their capabilities and limitations provides key insights that empower you to confidently decide which one to use and prepare you to navigate its shortcomings.
That is why today we are going to review the differences between traditional message brokers like RabbitMQ and modern event streamers like Apache Kafka. We'll look at how these event routers work, their respective advantages and drawbacks before considering their better-fit use cases.
This post was written alongside our other event-driven architecture articles, Introduction to Synchronous and Asynchronous Processing and Understanding Event-Driven Architecture and Serverless Opportunities
Earlier versions of event-driven architectures relied on message queues to transmit messages between event producers and event routers. Some examples of common message queues include RabbitMQ and Redis.
These traditional message brokers are persistent message frameworks that fuel the flow of events in an application's event-driven architecture. With traditional message brokers, the event router is intelligent while the event consumers are more reactive than proactive. This is typically referred to as the smart broker/ dumb consumers dynamic.
When using RabbitMQ, a popular open source message broker, events can be pushed directly to queues or to exchanges of the message broker. Developers subscribe event consumers to queues and can route messages from the exchanges to queues. These event flows are the powerful routing capabilities that make RabbitMQ so famous.
These event flows guarantee persistent messages in an event-driven architecture, which makes traditional message brokers great for use cases where you need to acknowledge individual messages.
At the same time, these flows lead to highly coupled integrations between the message broker and the event consumers. This tight-coupling leads to horizontal scaling issues, which is problematic when your app's traffic increases.
Another issue with tight coupling is that you will need to create an integration between each new event consumer you add to your app. These additional integrations can result in complex routing logics that could complicate your application's architecture and future deployments.
RabbitMQ is a traditional message broker with an established community and is celebrated for its ease of use. Additional advantages include:
A summary of the challenges to keep in mind when considering RabbitMQ:
The advent of event streaming message brokers like Apache Kafka transformed event-driven architecture and its possibilities.
Yes, both message brokers like RabbitMQ and event streaming routers like Apache Kafka receive events and pass them along to consumers, but how they do this is very different.
Apache Kafka is an open-source event streaming platform that keeps a recorded log of the events that pass through it and distributes this log to the services and event consumers in the application. These recorded and distributed logs are known as topics.
With Kafka, event producers emit events to topics, which store and organize the events in the order they occur. Event consumers listen to these topics and process the events accordingly.
Topics can have multiple event producers and event consumers, a feature that allows apps to scale flexibly. Moreover, topics are immutable records that exist as long as the use case requires. Unlike traditional message brokers where the messages are deleted once they are received by a consumer, event streams retain records of events, meaning events can be replayed whenever it is necessary to rebuild an app's state, to integrate a new feature, or even to correct errors.
As mentioned earlier, deciding which event router to use will depend on your use case and context. A simple question you can ask yourself when deciding which event router to use in your event-driven architecture:
If the answer is yes, RabbitMQ could very well be a better fit for your use case. If scaling is important and you need a higher throughput than RabbitMQ can deliver, Apache Kafka may be the better choice.
|Message Queues||Event Streamers|
|Scenarios where per message control is necessary,||Real-time analysis and data-processing,|
|Specifically defined integrations between producers and consumers,||Activity Tracking,|
|Traditional publish-subscribe systems,||Event Sourcing,|
|Long-running asynchronous tasks.||Log aggregation.|
Regardless of which event router you use to power your event-driven architecture, you can maximize the benefits of an event-driven architecture by deploying your app on a serverless platform.
In Understanding Event-Driven Architecture and Serverless Opportunities, we discussed the advantages of pairing event-driven architectures and serverless technology.
The two key takeaways:
Koyeb is a developer-friendly serverless platform to deploy apps globally. Thanks to native support of popular languages and built-in Docker container deployment, you can use Koyeb's serverless platform to deploy your: web apps, APIs, event-driven functions, background workers, web services, cron jobs.
With Koyeb, you can scale like internet giants without their budget. See the benefits of going serverless, get started with a free account today!
Here are some useful resources to get you started: