Message Broker – What’s It For?

A message broker is software that enables applications, systems, and services to communicate with each other and exchange information.
— IBM

As the name suggests, a message broker functions as an intermediary for messages (in this case, data/payload).
The payloads typically used are strings and JSON, depending on the message broker employed.

Before diving into case studies, let’s explore how application architecture works with a message broker.
Based on VMware, here’s a low-level system architecture design when using a message broker:

If we analogize this to a coffee shop:

• The cashier is the publisher who takes orders.
• The order queue monitor is the message broker.
• The barista is the consumer who prepares orders based on the queue.

Now, let’s move on to its applications:

1. Preventing Failed Transactions

Sometimes, an application/service responsible for processing transactions may suddenly go down, causing transactions to fail.

In this case, implementing a message broker after the request enters the API Gateway (publisher) ensures that the payload is sent to the message broker and then to the transaction service (consumer).
The data in the message broker is retained based on its retention configuration.

So, even if the consumer is down, the transaction isn’t lost and will be processed once the consumer is back up.

2. Broadcasting Data

This is akin to subscribing to a YouTube channel named ABC123.
All subscribers receive notifications when there’s new activity from that channel.

The producer publishes data to the broker, and all consumers subscribed to the broker receive the published data.

In this scenario, when a service experiences a data change, it simply publishes the data to the broker, and other services subscribed to the broker receive and update their respective storage.

3. Implementing Event-Driven Architecture

An event-driven architecture uses events to trigger and communicate between decoupled services and is common in modern applications built with microservices. An event is a change in state, or an update, like an item being placed in a shopping cart on an e-commerce website.

Here’s a simple example of an event-driven architecture flow in a mobile banking transfer process:

After the API Gateway receives a user’s request, the entire communication process is carried out asynchronously through the message broker, from the Transfer Service to the Notification Service.
Because the process is asynchronous, sometimes after making a transfer, our balance doesn’t immediately decrease, and the transaction status remains pending.

Okay, that’s all from me about message brokers. Hope it’s useful. Thank you! 👋