Get on board of a Flight into IoT 2 – Long-Haul – Microsoft Tech Days UK

Update: Recordings are done, it was a Blizz! Check out the recording here.

—

After the very successful flight Into Azure IoT from last year, a new flight will take off on November 24, 2022: A Flight into IoT 2, the Long-Haul:

We have a lot of upgrades: a new route, new telemetry, new Azure services, etc.

This is an online event with live recordings. Please register here for free.

Advertentie

MS Ignite 2022, IoT-related sessions and more

Yes, MS Ignite 2022 edition is around the corner again.

This year, there are both online sessions and in-person events, both in Seattle and selected locations all around the world:

The registration is opened already and the session planner is available.

Here is a selection of IoT-related sessions, together with some sessions I like personally.

Flex your Big Data detective skills with the Kusto detective agency challenge

Flex your Big Data detective skills with multiple cases of the Kusto detective agency challenge.

Solve puzzles using the Kusto Query language and earn awards.

Are you interested?

Azure Data Explorer connector for Blob storage (IoT Hub) files

Over the last couple of months, I have written several blog posts regarding Azure Data explorer ingestion connectors.

I checked out the IoT Hub connector, dynamic mapping, and table update policies.

Until now, this is all based on both the IoT Hub connector and the Event Hub connector:

Next to those two, Azure Data Explorer also supports ingesting complete files using the third connector: Blob Storage ingestion in combination with EventGrid events:

Let’s check out what this connector offers and how to set it up.

As we will find out, it even has a surprise for those of you ingesting IoT Hub messages.

Exploring full Azure IoT Hub device support using MQTT(s) only

Most of you Azure IoT developers are connecting devices to the Azure cloud using the Azure IoT Device SDKs.

Using these SDKs, you can connect a device to the cloud in an easy and secure way with your favorite programming language like C#, C, Java, Python, or Node.js.

This is the recommended way because it offers a convenient and optimized way to support all Azure IoT Hub features like Device Twin, Direct methods, and Cloud messages. It takes away a lot of the code wiring and you can focus on functionality.

Still, in a few instances, like working with very constrained devices, there could be a need for bare MQTT support:

MQTT is the de-facto standard for stateful communication in the IoT World (btw. Bare AMQP is offered too).

Let’s see how the Azure IoT Hub supports bare MQTT.

The risk of pinning TLS certificates in IoT solutions and why your awareness is needed

Do you have cloud-connected IoT devices to the Azure IoT Hub and have you specified the specific public ‘Baltimore’ TLS certificate currently in use by the IoT Hub? Do you know customers having (low-powered) devices connected to Azure IoT and possibly pinned that certificate?

Then, please be aware of the following message.

Extending the AZ-220 Digital Twins hands-on lab with 3D visualization

Azure Digital Twins is advertised as a “platform as a service (PaaS) offering that enables the creation of twin graphs based on digital models of entire environments, which could be buildings, factories, farms, energy networks, railways, stadiums, and more—even entire cities”.

This sounds promising but it does not really ring a bell, does it?

Fortunately, besides the excellent documentation, Microsoft provides a great learning path in MS Learn as part of the AZ-220 Azure IoT developer exam preparations.

There, you will learn how Azure Digital Twins offers new opportunities for representing an Internet of Things solution via twin models, twin relations, and a runtime environment.

You finish the learning path with a hands-on lab where you build a model around a cheese factory and ingest sensor telemetry:

In the demo, the telemetry flows through the runtime and ends up in Time Series Insights.

Yes, the learning path is a good start and will prepare you for the exam or the assessment (you need to pass this assessment for a free one-year certification renewal).

On the other hand, many extra features could be added to turn this good start into a great start!

Think about propagating Azure Digital Twins events and twin property changes through the graph and visualizing live updates of twins in a 3D model, complete with alerts.

Let’s check out some of these additional features and see what you need to do to extend the ADT example.

Using ADX table update policies projecting raw messages to target tables

I have blogged about Azure Data Explorer (ADX) integration already a couple of times.

The new ADX database data connection for IoT Hub is a nice addition but appears to be quite static because you can only fill in one (single) table mapping.

ADX also supports dynamic routing (using that same data connection) where the message body is accompanied by application/user properties which tell ADX the table and mapping to use during ingest.

The ultimate goal is to have a solution where one stream of data, with one or more types of messages, is split into multiple tables:

The combination of IoT Hub routing and multiple endpoints should help with that but due to message enrichment limitations, we need to find another way.

A viable solution is the ADX table update policy.

Let’s check it out.

How to cope with IoT Hub enrichment restrictions

As seen in my previous post, The IoT Hub routing feature supports message enrichment, both for IoT devices and IoT edge modules.

Using the routing message enrichments, each incoming message gets extra user properties based on either static values, device twin tags, or device twin desired properties.

Unfortunately, only ten enrichments can be added:

If you want to pass more values, this will not work for you.

It would be great if nested JSON properties would count as one.

Again, unfortunately, only simple types (string, decimal, boolean, date/time, etc.) are supported so this excludes nested JSON (complex types).

Below, a viable solution to overcome both restrictions, using Azure Stream Analytics, is presented.

Let’s see how this works out.

Dynamic routing of IoT Hub telemetry to Azure Data Explorer

Azure Data Explorer (ADX) is a great data exploration tool for IoT developers building a full IoT solution. This could be a perfect target for the cold path.

As seen in my previous blog post, ADX even offers a native connector for the IoT Hub. This is based on the ‘default EventHub compatible endpoint’ offered by this cloud IoT gateway (optionally the built-in Events endpoint in the routing section of the IoT Hub or using the fallback mechanism).

Most of the documentation regarding this ADX connector is following this ‘happy flow’ where one connector stores incoming IoT telemetry in one ADX table using static routing.

This is a serious limitation where most IoT Hubs ingest multiple types of messages. These will not fit into that single table.

Luckily, the connector also offers the possibility to allow routing to other databases:

Here, we will check out this dynamic routing option and see how this provides much more flexibility.