Azure Stream Analytics anomaly detection on the edge

Back in 2018, the Azure Stream Analytics team announced Anomaly detection for Azure Stream Analytics. And, it was also supported on the Azure IoT Edge. In November 2018, I was allowed to demonstrate this at the SPS IPC Drives in Nuremberg:

Since then, anomaly detection has become a first citizen of Azure Stream Analytics.

Azure Stream Analytics Anomaly Detection is able to ‘automatically’ detect spikes, dips, and trends in a stream of values. This is based on math and all you need to do is to specify how many values you expect and how sensitive the detection must be. It is in fact Machine Learning, in the end. So it is a prediction with a certain certainty…

And it’s just part of the Stream Analytics query language. So on the edge, we can deploy a Stream Analytics module (the engine of stream analytics). This is a fixed module from Microsoft.

All you have to do is feeding it a query, inputs, outputs and, user-defined functions if available:

We can define all this in the cloud, inside a stream analytics job and we can even test it. Those parts are then packed as a blob and put in blob storage. The Stream Analytics job can then download it and run it.

The inputs and outputs of the Stream Analytics job can be attached to the normal Azure IoT Edge routing mechanism.

Let’s see how this is set up:

IoT Plug and Play, modeling IoT Central devices

If you have built an IoT Device yourself and are finally able to send telemetry to the cloud, you should be familiar with the scenario where you have to repeat the hard work of describing the messages all again on ingestion.

IoT Devices expose D2C telemetry and it can also support C2D communication. This interface is most of the time unique for that device. To be able to get insights from a device you have to be able to react to its interface.

Wouldn’t it be nice if a device was able to provide metadata about its interface once it connects to the cloud? This way, the incoming D2C telemetry could automatically result in e.g. a full user interface. And all C2D output could be represented by pre-configured input controls.

With IoT Plug and Play this is all possible:

IoT Plug and Play enables solution builders to integrate smart devices with their solutions without any manual configuration. At the core of IoT Plug and Play, is a device model that a device uses to advertise its capabilities to an IoT Plug and Play-enabled application

https://docs.microsoft.com/en-us/azure/iot-pnp/overview-iot-plug-and-play

Microsoft provides a way to describe the interface of a device and annotate every feature.

To experience how this works, we will look at the best example: Azure IoT Central.

This SaaS IoT Dashboard makes perfect use of the ability how devices can expose their meta-model.

Let’s see how.

Add local storage to Azure IoT Edge modules using Docker Bind

Azure IoT Edge makes use of the Moby container runtime so IoT Edge modules (being Docker containers) can work together and offer logic on the edge.

Docker containers are ‘sandboxed’. This means that the logic within the containers has limited access to the environment they ‘live’ in.

By default, containers have no SUDO rights, no access to the host filesystem, and just limited network capabilities.

Though, containers can be granted elevated rights. One of these is the right to access the filesystem.

In this blog, we will see how to configure a container with access to the filesystem. To demonstrate this, a custom IoT Edge module is introduced, an IoT Edge filewatcher for CSV files:

Handling Advantech Wise 710 OPC-UA telemetry using OPCPublisher

Microsoft has an extensive IoT platform based on Azure.

It provides so many features, this can be overwhelming for customers. Therefore, Microsoft provides Azure IoT solution accelerators, based on the Azure IoT reference architecture.

Some of the original accelerators (it started with Azure IoT suites) like Remote Monitoring are now outdated or even archived. These are replaced by excellent Azure IoT Central apps which demonstrate the capabilities of the IoT platform for numerous markets and verticals:

There is still one original accelerator alive-and-kicking: the Connected Factory. This one demonstrates the use of OPC-UA protocols on the edge and in the cloud.

More than two years ago, I already wrote about this accelerator and the OPC publisher module, the backbone of this accelerator. Since then, a lot has changed. Some functionality is (temporarily) deprecated so I got a lot of questions based on the old blogs.

So it’s time to update it a little and see how the OPC Publisher is doing these days.

The OPC-UA solution is open source and covers a lot:

I limit this blog to the scope of my very first blog, extracting OPC-UA messages and send them to the cloud using a ‘published nodes’ file.

We just need an OPC-UA server to get some sample data from. For this, I used an Advantech Wise 710 as an industrial protocol gateway.

Azure IoT Edge module metrics in action

We are familiar with the Azure IoT Hub metrics which are offered. The Azure cloud tells us eg. how many messages are received or the number of devices that are connected.

If we look at Azure IoT Edge, you had to collect your own made metrics in the past.

Because IoT Edge modules are Docker containers and therefore sandboxed, you had to rely on the (third-party) logic to capture Host metrics. Regarding metrics about the edge agent and hub, these were not available.

Until now.

With the most recent IoT Edge runtimes, agent, and hub, we have access to Edge metrics.

Both the Agent and Hub module expose the metrics over HTTP endpoints:

Within the Moby runtime, port 9600 is exposed on both individual modules. Outside the runtime, we have to assign individual host ports to prevent using the same host port.

Let’s see how this looks like and how we can harvest metrics in a custom container.

Getting started with the Azure IoT Central Rest API

Azure IoT Central is a SaaS platform for IoT projects.

If you are looking for a way to manage and monitor your IoT devices outside the Azure Portal or are not able to build your own IoT platform, IoT Central is the place to be. And you can extend this portal with custom Azure resources using the export functionality.

All you need is to have browser access to Azure IoT Central. You can even run it for free for seven days to test it out. Also, the first two devices registered are free too.

Once you have worked with Azure IoT central, you have mastered it using the portal. If you want to scale up eg. the number of devices or users, automation of your tasks becomes necessary.

For this, Azure IoT Central offers a Rest API.

Let’s check this API out.

Counting the current number of IoT Hub messages and devices

Microsoft provides SDKs for both building IoT devices and accessing the IoT Hub (based on several programming languages).

Regarding the IoT Hub, with the SDK you are able to eg. register and control devices.

Still, some actions are possible with the IoT Hub which is not supported by the SDK.

For those actions, Microsoft supports an IoTHub Rest API which extends the capabilities of the SDK.

In the past, I demonstrated how to work with C2D messages using the REST API.

A few days back, I came along with this MS Learn question about IoT Hub message counts. This also has to be tackled by the REST API.

Let’s see how.

Using a Weidmueller UC 20 Controller as Azure IoT Edge child device

Azure IoT Edge is a powerful solution for your edge computing needs. It can collect telemetry, make local decisions, and send data to the cloud. This works great if an internet connection is available. If the connection is temporarily broken, everything still works. The telemetry is temporarily persisted so no data is lost.

An edge gateway can also act as a transparent gateway:

Here, child devices are made part of the local routing mechanism of the edge. The child devices are configured to send their telemetry to the edge device. From there, the same telemetry is sent to the cloud as if it’s sent by the child device itself.

The main advantages are:

1. If no internet connection is available, the child telemetry is stored on the edge until the connection is restored. The child devices have no notion of the edge gateway, hence ‘transparent’
2. The logic running on the edge is able to access the telemetry coming from child devices so this can be used and combined with other data to take local decisions

This architecture is also known as downstream devices.

I already wrote a blog on this topic previously. In there, some test apps stole the show.

Now, let’s see this in action with an actual industrial device. We also check out sending telemetry back:

We will be working with a Weidmueller UC20, an automation controller.

Expanding Raspberry PI I/O using I²C on Azure IoT Edge

The GPIO of a Raspberry gives you the opportunity to interact with the physical world using digital pins and various IO busses like SPI and I²C.

In the past, in this blog, I already demonstrated how to access the GPIO of a Raspberry Pi.

In the last few months, I spent my spare time building a beerlift:

The beerlift is capable to serve multiple bottles of beer so each bottle has its bottle holder:

The bottle holder contains a switch to detect a bottle being placed or being removed. It also contains a LED so it can visualize if a bottle is placed or removed or eg. advertised.

I wanted to support up to sixteen bottles (so 32 switches and LEDs) which exceed the GPIO pin limitation of a Raspberry Pi.

Therefore, I bought myself a couple of MCP23017 I/O Expanders. This device offers sixteen digital inputs or outputs over a serial interface. I went for the I2C version:

Let’s see how we can use them in an Azure IoT Edge solution.

Sending IoT Hub telemetry to a Blazor Web App

For those who are interested in software development for the web using the C# programming language, Blazor is a viable alternative for building progressive websites as compared to Asp.Net Core / Angular / JavaScript.

Blazor lets you build interactive web UIs using C# instead of JavaScript. Blazor apps are composed of reusable web UI components implemented using C#, HTML, and CSS. Both client and server code is written in C#, allowing you to share code and libraries.

In the past, I already implemented Blazor on the Edge, including message routing.

Now, let’s see how we can integrate a Blazor website with telemetry coming from an Azure IoT Hub in the cloud.

For this to happen, we need this architecture:

So, the moving parts are:

• An IoT Hub with message routing enabled
• Azure Function with IoT Hub / EventHub trigger
• Server-side Blazor website with API Controller integration

Let’s see how this is set up.