How to cope with IoT Hub enrichment restrictions

As seen in my previous post, The IoT Hub routing feature supports message enrichment, both for IoT devices and IoT edge modules.

Using the routing message enrichments, each incoming message gets extra user properties based on either static values, device twin tags, or device twin desired properties.

Unfortunately, only ten enrichments can be added:

If you want to pass more values, this will not work for you.

It would be great if nested JSON properties would count as one.

Again, unfortunately, only simple types (string, decimal, boolean, date/time, etc.) are supported so this excludes nested JSON (complex types).

Below, a viable solution to overcome both restrictions, using Azure Stream Analytics, is presented.

Let’s see how this works out.

Advertentie

Creating an Azure Stream Analytics job using VS Code

Azure Stream Analytics is often the centerpiece of our IoT solutions.

It acts like a rule engine where data streams from multiple sources can be combined together, even enriched with static reference data.

Azure Stream Analytics does not come cheap if you only want to do some simple aggregations. For that, probably Azure Functions can help you out.

But, when it comes to more elaborate rules using multiple inputs, multiple outputs, time windowing, custom functions, and Machine learning integration, Azure Stream Analytics (and many more capabilities) should be your first choice:

Normally, I demonstrate Azure Stream Analytics using the Azure Portal.

There, it offers me a convenient browser experience where I can show how inputs, outputs, and user-defined functions are created. I can also copy/paste a (basic) query and demonstrate how it can be tested and run in a simple manner:

For people new to Azure Stream Analytics, this is a perfect starting point.

Still, this is for demonstration purposes only!

The Azure portal lacks (professional) abilities like source control/versioning, superior user-defined functions, and diagnostics.

If you plan to use Azure Stream Analytics in your projects, please consider starting using the VS Code project template.

In this blog, we will see how to start with the Visual Studio Code project for Stream Analytics.

FutureTech 2022 announced, CFP now closed

Update: CFP is now closed. The event is rescheduled.

The Dutch FutureTech is a one-day event in The Netherlands. In 2022 it will take place on March 17 June 21, 2022 at the Jaarbeurs Utrecht.

This event is the Microsoft Technologies and .Net conference for the Netherlands showing the latest innovations using Microsoft technology.

Expect the best national & international speakers delivering quality content about the hottest IT topics, of course with the overall theme: Microsoft technologies.

Get on board for a Flight into Azure IoT

Update: It was a blizz! The airplane in Budapest after a successful flight. landed The recording is available here:

On Monday, November 15th a great team of MVPs will walk you through the simple process of setting up an Azure IoT solution.

What to expect?

We will take you through an end-to-end solution from real flight tracking data like speed, altitude and location from a flight departing London. During the event we’ll track the flight’s progress to its destination and show how to get that data from an IoT device at the edge into Microsoft Azure IoT and the cloud where it can be processed for display on a dashboard or stored for later processing. We won’t be able to teach you how to become an IoT expert in the 2.5 hours we have – but what we can do is show you how to build on your current developer skills to integrate IoT into your business applications (and passion projects!) and set you up on your journey to become certified in IoT with the Microsoft AZ220 qualification.

I myself will show the power of Time Series Insights to capture data, let you understand how that data can support business objectives, and show how to surface that data from an engineer’s perspective.

The interactive, online, event takes place at 10:00-12:45 GMT (UTC±0).

I hope to see you there!

Please register at aka.ms/techdays/iot

Microsoft Build 2020 IoT sessions and more

During the recent Build event, a lot of news about Azure IoT announced.

Here is a summary of all online available IoT sessions and workshops, plus some personal favorites from which I think this is interested for IoT Developers.

Inter-module messaging without IoT Edge routing

Azure IoT supports cloud-to-device messaging with Direct Methods. This is an important tool when you want to control your devices real-time or if you want to execute logic on the device real-time.

In the past I have written about Direct Methods a couple of times. like this blog. I then wrote about IoT Edge supporting Direct Methods too. To be more specific, Modules in an IoT Edge support Direct Methods.

But my colleague Heindirk pointed me at a little gem unknown to me.

The same logic used to communicate from Azure cloud to a module can also be used to communicate from one module to another module without IoT Edge routes!

This makes several new IoT strategies possible!

Let’s check out this gem.

IoT Edge group enrollments using symmetric keys

In my previous blog about using a VM as IoT Edge device, it became clear that this could be used for testing IoT Edge at scale.

Testing IoT Edge at scale means testing device enrollments using the Device Provisioning Service (DPS) and IoT Hub deployments at scale.

We will look at both situations. But before we check out a group enrollment, first we look at an individual enrollment, just for comparison.

We will use the recently announced IoT Edge support for symmetric keys in DPS.

Tracking the location of your IoT Central devices

Recently IoT Central added location tracking.

In the past, it was already possible to add a location to your IoT Central devices. And these locations were shown on a map. But these locations were fixed, part of the device template properties. So it was only present in the metadata.

But now we can pass a location in the telemetry which is produced by your device.

Let’s see how it’s done and how the location is integrated into the various IoT Central dashboards.

When your clone is several commits behind the master

Once in a while, I clone a project just to check out the code, have it compiled or even to request a pull for an update.

That’s ok.

But what if my clone gets stale? What if it is several commits behind the original repository? You see a message like this:

How do you fix it? Is there a hidden button somewhere in the GitHub portal?

Event Hub alerts, great for detecting drops in connectivity

If we look at the Azure IoT Reference Architecture we see how streaming data is the heart of the IoT platform:

Data arrives at the IoT Hub and can be routed to any Azure resource using eg. the IoT Hub routing, Stream Analytics jobs and Event Hubs.

This gives us the maximum flexibility to divide the data into three major data streams or storage:

1. Hot path – Event, Alert, Conditions
2. Warm path – aggregated data, data for reporting
3. Cold path – the raw data, mainly untouched and available in large quantities; great for data scientists

But there is an often unseen, even ignored forth stream. And that is the stream of data for monitoring.

In my earlier blogs, I wrote several times hoe heartbeats en watchdogs can tell the story about the quality of the data. And I have shown how eg. Azure Functions and notification services can improve the insights about the quality of the communication.

Today I want to add a little gem to that list.

Let’s check out Monitoring Alerts in Event Hubs. It’s part of the overall Monitoring service in Azure.