Flexible message routing in Azure IoT Hub using C#

The Azure IoT Platform is a very versatile solution for all your IoT needs. Azure supports multiple resources for storing large amounts of data, querying immense streams of data coming in, having event buses which can hold millions of messages, serverless functions, reporting and Machine Learning; what more do you need?

But it all starts with the IoT Hub:

“Azure IoT Hub is a fully managed service that enables reliable and secure bidirectional communications between millions of IoT devices and a solution back end”

Normally, whenever I start a new IoT Platform solution in Azure, I start with an IoT Hub and connect it to a Stream Analytics job as an input source. Messages arriving at the IoT Hub are then passed directly into the Stream Analytics job to be examined. And the Stream Analytics job can pass some or all messages (or transformed messages) to multiple output sinks eg. Event Hubs, Service Bus Queue or Topic, Blob Storage, etc.

The arriving messages carry telemetry information from the device. But what if the messages are sent in a certain context? What if a message has a high or low priority? Should we pollute the message with this ‘routing’ information? And Act on it inside the Stream Analytics job?

A few week ago, Microsoft introduced a new feature in IoT Hub, called message routing.

This makes it fairly easy to react on difference messages, arriving at the same IoT Hub, but intended to handled differently. Routing is perfect for this matter. We can declare extra endpoints directly in the IoT Hub. And depending on message properties, messages can be sent directly to these endpoints:

There are two important things to keep in mind. First, the message properties are extra annotations (defined by the IoT Hub device client), we are not peaking inside the message itself. And second, messages can still end up in a Stream Analytics input when it is ignored by the active routes.

Let’s take a closer look.

Update 2017-06-01: Microsoft announced that routing is now supporting values from the actual telemetry, great news and now it’s more intuitive!

First of all, the number of endpoints and routes are almost the same for each IoT Hub tier. Only the free tier supports just one extra endpoint and just up to five routes. Fair enough, it’s free:

• F1 free –> 1 endpoint + 5 routes
• S1 standard -> 10 endpoints + 100 routes
• S2 standard -> 10 endpoints + 100 routes
• S3 standard -> 10 endpoints + 100 routes

Note: Billing for S3 is approx. 155 euros per day, with a minimum of one day. I will lock your MSDN Azure credits in just one day!

Go to the IoT Hub panel to see the new features:

First, let’s create some endpoints. Without endpoints, we can not define meaningful routes.

An endpoint can either be a Service Bus queue, a Service Bus topic or an Event Hub.

I have already defined two Service Bus queues, name ‘queueone’ and ‘queuetwo’. Let’s route messages to these two endpoints. This is how we create endpoints:

As you can see, the ‘original’ built-in endpoint is still available in the overview:

Now, let’s create a route:

We have defined a route for ‘queueone’. If a message is annotated with a ‘severity’ of ‘high’, the message is routed to this queue. The route can be tested using the JSON at the bottom. As you can see, the annotation matches the rule.

Annotations are passed as strings. So if you want to use numbers, you will have to convert the text:

And the rule language also supports case sensitivity:

A more complex rule is also possible:

And it’s even possible to check if a certain property is available, using the ‘IS_DEFINED’ function:

So we have seen several ways to write a query. Full documentation about the query language can be found here.

In the end, we have multiple rules. And as you can see, I added two:

Note: adding a rule takes a moment. Please be gentle, check the logging to see if the first rule is added before you add the next one. Things could lock up.

We can also test a message annotation across all active rules (rule can be disabled and enabled again):

As you can see, the same message can apply to multiple endpoints. You can mix and match as you like.

And if no rule is matched, the message will be sent to the fallback endpoint (if enabled):

Sending a message

We have seen how we can manage message routing in IoT Hub. But how do we send a message? Let’s look at the code, sent by a UWP app.

First, we define the message to be sent:

public class Telemetry
{
    public int errorCode { get; set; }

    public int numberOfCycles { get; set; }
}

The message has no knowledge about routing (the annotation will be done outside the message) so when messages are sent by the device:

The messages end up in a blob storage, defined as output sink for Stream Analytics, the fallback endpoint:

So let’s annotate:

public static async Task SendDeviceToCloudMessageAsync(Telemetry telemetry)
{
    var deviceClient = DeviceClient.CreateFromConnectionString(deviceConnectionString, TransportType.Amqp);

    var message = new Message(Encoding.ASCII.GetBytes(Newtonsoft.Json.JsonConvert.SerializeObject(telemetry)));

    <strong>message.Properties.Add("severity", "high");</strong>

    await deviceClient.SendEventAsync(message);
}

The annotation is done using the properties property, and a property is a combination of a key and a string value.

When I sent these ‘severity’ messages, the Device Explorer does show the message, including the annotation:

Note: I did not always see the messages appear once the annotation was added. Just try it out.

And the good news is that the messages do not end up in the blob storage too! So message routing has done something. The message should end up in both queues.

We can inspect the service bus queues using a tool called ServiceBusExplorer. Unfortunately, this tool is only available as code, you have to zip the code from Github and compile it yourself.

Update: Recently I discovered https://www.servicebus360.com. It has a livetime free account. This simplifies checking out the queues.

Connect within this explorer with the service bus (supplying the connection string is all we have to do) and navigate to the queues.

I have also sent a message with severity low. This is what I expect: two messages in Queue Two; three message in Queue One. This is because Queue One takes all messages annotated with ‘severity’ and Queue Two only takes ‘high’ and ‘medium’ messages.

What I found in Queue One:

There are three messages, and that’s exactly what was expected.

And what I found in Queue Two:

Only the two messages with high and medium priority end up in this queue. The message routing in the IoT Hub has done its job.

Conclusion

IoT Hub message routing is a powerful tool for splitting messages based on an extra annotation.