Using the new IoT Hub CosmosDB routing endpoint

The Azure IoT Hub is your cloud gateway for ingesting telemetry.

The IoT Hub cannot persist incoming messages so these must be forwarded to other Azure services.

Traditionally, the messages are exposed over an Event Hub-compatible endpoint.

More recent, (non-functional) IoT Hub routing is added where specific Azure services can be connected as an endpoint:

At this moment we can define:

• The build-in endpoint (to keep the original way of distributing messages)
• Event Hub
• Service Bus (Topics/Queues)
• Storage account, blob storage (perfect for cheap cold storage)

Lately, a native endpoint for CosmosDB has been made available.

This takes the pain away of having to set up extra resources between the IoT Hub and CosmosDB, just to transport messages from one resource to another. This is mostly done using a Stream Analytics job or custom Azure Functions.

In this post, let’s check out the new endpoint.

Advertentie

How to cope with IoT Hub enrichment restrictions

As seen in my previous post, The IoT Hub routing feature supports message enrichment, both for IoT devices and IoT edge modules.

Using the routing message enrichments, each incoming message gets extra user properties based on either static values, device twin tags, or device twin desired properties.

Unfortunately, only ten enrichments can be added:

If you want to pass more values, this will not work for you.

It would be great if nested JSON properties would count as one.

Again, unfortunately, only simple types (string, decimal, boolean, date/time, etc.) are supported so this excludes nested JSON (complex types).

Below, a viable solution to overcome both restrictions, using Azure Stream Analytics, is presented.

Let’s see how this works out.

Azure IoTHub routing revisited, Blob Storage Endpoints

Recently, Microsoft added some extra features to the IoTHub routing abilities:

1. Support for routing using the message body
2. Support for Blob Storage as endpoint

In this blog, we will look at both features using the Visual Studio 2017 extension called the IoT Hub Connected Service, which is updated also.

But first, let’s look at the new Blob Storage endpoint.

Sending telemetry to a Blob Storage container is a simple and efficient way for cold path analytics:

Until recently, there were a few ways to do this:

• Sending IoTHub output to a Stream Analytics job, which filled some blob
• Sending IoTHub output to an Azure Function, which filled some blob
• Making use of the IoT Hub ability to receive blobs

The first two ways can be done using extra Azure resources so additional costs are involved. The third one is only used in very specific circumstances.

The new Blob Storage endpoint is a simple but very powerful way of making Cold path analytics available, right from the IoTHub.

Introduction to the IoT Edge SDK V1, part 4: IoTHub routing

Update: Recently, Microsoft introduced the new V2 version of their IoT Edge. This blog is referencing the former but still supported V1.

We have already made great progress understanding and using the Azure Gateway SDK.

What do we have right now? We can send telemetry data from multiple ‘offline’ devices and accept commands from the IoT Hub.

The data we send is well-formatted JSON so we are good to go.

But I am a bit worried. While reading all documentation regarding the transformation from Azure Gateway SDK towards the IoT Edge SDK, it is clear that multiple types of messages are sent to the IoT Hub. For example, I can imagine that a Stream Analytics module generates other data.

And let’s look at a more ‘close to earth’ example. The gateway itself is a potential device too! But I do not want to mix data coming from the gateway and from sensor devices.

Of course, we recently got the ability to route messages using the message sent. But what about using the properties? This keeps the message content clean.

Will this be working?