Event Hubs Scalability
Event Hubs is a streaming and event ingestion service on the Microsoft Azure Platform capable of receiving and processing millions of events per second. The service will allow you to process and analyze the massive amounts of data produced by your connected devices and applications.
In the tip availability and consistency, we discussed partitioning in an Event Hub. Partitioning also plays a role in scalability of Event Hubs together with Through Put units. The latter is a pre-paid unit of capacity, i.e., you specify the number of throughput units when provisioning a namespace. Furthermore, through scale setting in the namespace, you can alter the number of throughput units.
According to the Microsoft docs, a Throughput Unit (TU):
• Ingress: Up to 1 MB per second or 1000 events per second (whichever comes first).
• Egress: Up to 2 MB per second or 4096 events per second.
And with Auto-inflate enabled the scaling is done automatically for you up to the specified number of TPUs.
Note that TPUs are set on namespace level, hence when you specify (buy) one TPU and have, for instance, five Event Hubs your increasing usage of all of these Event Hubs can be 1TPU. Individually, each partition in an Event Hub can handle a throughput of 1 MB/Sec. Hence, the number of partitions can determine how fast and in parallel you can receive, and process events based on the number of TU you set on the Event Hub Namespace.
Let’s assume you have set the number of TUs on six and auto-inflate to 12 (see screenshot above), and you have 2 Event Hubs each with four partitions, then the maximum throughput in a partition is 4 MB/Sec. The Egress is two times the Ingress hence 8 MB/ Sec. Note that you can take advantage of more TPU than you have partitions!
When building your solution, think about scaling and start with a low number of TPUs first and slowly depending on the workload increase the TPUs. That said you still need to think about the partitioning of your Event Hubs as they cannot be altered later on.