Flexible Event Management with Azure Event Grid Dynamic Event Schemas

Microsoft Azure Event Grid supports dynamic schemas for event publishers that enable them to dynamically modify the payload structure without requiring subscribers to update their event subscriptions. It allows event-driven systems to evolve without significantly altering the infrastructure.

This is especially useful for event-driven systems that deal with large amounts of data since it enables event publishers to add, remove, or change the structure of their event payloads based on their changing needs. For example, a publisher may want to add a new field to their event payloads to provide additional information to subscribers, or they may want to remove a field that is no longer needed. Maintaining and evolving event-driven systems using dynamic schema is more accessible than updating subscribers.

The Azure Event Grid supports dynamic schema for all event types, including custom and cloud events, offering developers greater flexibility and ease of operation.

Demo

Start by creating a new Event Grid topic using the following command:

az eventgrid topic create -g eventgrid-rg --name students -l australiaeast   

After creating the topic, you should follow the steps outlined in the Microsoft documentation website to create an Azure storage queue subscription.

Here is a screenshot of the subscription I created through Event Grid's user interface:

Figure 1 Create a storage queue subscription for the Event Grid topic

Then, using your preferred IDE, create a console application and install the following NuGet packages:

$ dotnet add package Azure.Identity 
$ dotnet add package Azure.Messaging.EventGrid 

You can now publish events with different schemas to the Event Grid topic you created using this code:

using Azure.Identity; 
using Azure.Messaging.EventGrid; 

// Define the Event Grid Topic for the custom event
var eventTopic = "<your-event-grid-topic-url>"; 

// Create an Event Grid client 
var client = new EventGridPublisherClient(new(eventTopic), new DefaultAzureCredential()); 

// Define the schema version of the custom event 
var schemaVersion = "1.0"; 

// Define the data for the custom event 
dynamic customEventData = new { FirstName = "Rahul", LastName = "Rai" }; 

// Publish the custom event to Event Grid with schema version 1.0 
await client.SendEventAsync(new EventGridEvent("Custom Event Subject", "CustomEvent", schemaVersion, customEventData)); 

Console.WriteLine("Published custom event with schema version 1.0 to Azure Event Grid."); 

// Update the schema version of the custom event to 2.0 
schemaVersion = "2.0"; 

// Update the data for the custom event 
customEventData = new { FirstName = "Rahul", LastName = "Rai", Standard = 7 }; 

// Publish the custom event to Event Grid with schema version 2.0 
await client.SendEventAsync(new EventGridEvent("Custom Event Subject", "CustomEvent", schemaVersion, customEventData)); 

Console.WriteLine("Published custom event with schema version 2.0 to Azure Event Grid."); 

In this example, we first created an instance of the EventGridPublisherClient class, which is used to publish events to Azure Event Grid. Then we defined the Event Grid topic endpoint, where those events will be published.

Next, we defined the custom event data schema version, a string indicating the version of the event's schema. In the example, we set the schema version as "1.0". We also defined the custom event data, a JSON object containing the properties FirstName and LastName.

The custom event was then published to the Event Grid topic using the SendEventAsync method of the EventGridPublisherClient class. The custom event is specified as an instance of the EventGridEvent class, which has a DataVersion property that specifies the schema version of the event data.

After publishing the event, we updated the event data schema and published another event.

The following screenshot shows the output from the console application after the events have been published:

Figure 2 Output from the console application

Here is a screenshot of the events received by the queue configured to subscribe to the topic.

Figure 3 Events received by the storage queue