Use Consumer Groups in Azure Event Hubs to support multiple subscribers

Azure Event Hubs is a scalable event processor that ingests high volumes of events and data, with low latency and high reliability. It is often used in big data and IoT solutions.

Often times, developers have questions about pub-sub messaging in Azure. While pub-sub messaging is, typically, better addressed with Service Bus Queues and Topics, Event Hubs do have support for multiple subscribers on the same data stream.

To address multiple subscribers in Azure Event Hubs, a developer needs to create multiple consumer groups, with each providing isolation from other receivers reading from the same stream.

When an Azure Event Hub is created, a $Default consumer group is created. This allows an application using that consumer group to get started and begin reading events. If we want to add more readers to this stream, we need to create a new Consumer group from the Azure Portal.

We now need to provide a name for this consumer group.

With our consumer group now created, we need to configure a subscriber to take advantage of this consumer group.

To demonstrate how we can configure a consumer group as a subscriber, we will use Azure Logic Apps and create an Event Hub trigger. There is a drop-down half-way down the list called Consumer group name. Here we will find both our $Default and newly created ordersreceiver consumer groups. In our scenario, we use the ordersreceiver consumer group which will process messages independently than $Default.

Event Hubs support multiple subscribers when reading event streams. This approach allows multiple readers to consume the same data stream and separate concerns for downstream applications.