Azure Data Explorer as a data source for Azure Managed Grafana dashboards

Internet of Things solutions must provide insights, based on data provided by devices and other resources.

Adding dashboards to an IoT solution is the most common way to add that value.

There are a number of popular platforms for custom dashboarding, including Grafana.

Their solution is offered both in a commercial way and as an open-source version.

I have already written a couple of blog posts about working with Grafana, both in the cloud and on the edge.

The cloud version was initially based on VM technology in Azure.

Recently, Azure started offering a managed PaaS version. It takes away the ‘pain’ of managing the environment.

Due to security concerns, Azure Managed Grafana doesn’t currently support custom plugins. According to the website, you’ll have the option to add Grafana Enterprise plugins at a later date.

Personally, Grafana is a nice addition for me to display data stored in Azure Data Explorer next to the dashboarding solution offered by Azure Data Explorer itself (currently in preview).

So, there is now an integration possible between these two worlds.

Let’s check out how Grafana dashboards can be put on top of Azure Data Explorer.

Advertentie

Extending the AZ-220 Digital Twins hands-on lab with 3D visualization

Azure Digital Twins is advertised as a “platform as a service (PaaS) offering that enables the creation of twin graphs based on digital models of entire environments, which could be buildings, factories, farms, energy networks, railways, stadiums, and more—even entire cities”.

This sounds promising but it does not really ring a bell, does it?

Fortunately, besides the excellent documentation, Microsoft provides a great learning path in MS Learn as part of the AZ-220 Azure IoT developer exam preparations.

There, you will learn how Azure Digital Twins offers new opportunities for representing an Internet of Things solution via twin models, twin relations, and a runtime environment.

You finish the learning path with a hands-on lab where you build a model around a cheese factory and ingest sensor telemetry:

In the demo, the telemetry flows through the runtime and ends up in Time Series Insights.

Yes, the learning path is a good start and will prepare you for the exam or the assessment (you need to pass this assessment for a free one-year certification renewal).

On the other hand, many extra features could be added to turn this good start into a great start!

Think about propagating Azure Digital Twins events and twin property changes through the graph and visualizing live updates of twins in a 3D model, complete with alerts.

Let’s check out some of these additional features and see what you need to do to extend the ADT example.

Getting started with Azure IoT Plug and Play

Microsoft is the founder of this concept called Azure IoT Plug and Play:

IoT Plug and Play enables solution builders to integrate smart devices with their solutions without any manual configuration.

The idea is that the device describes itself using some identification key. This key, the Model ID (or DTMI, Device Twin Model Identification), is bound to a complete model written in DTDL (Digital Twins Definition Language).

Using this model, the interface (or capabilities) of this device can be read:

• Properties (Azure IoT Device Twin desired properties and reported properties)
• Telemetry (the D2C messages)
• Command methods (based on Azure IoT Direct methods)

Once a device starts communicating using this deterministic interface, a User Interface can be provided dynamically.

This is the same principle of Plug and Play devices like a mouse or a webcam. If you plug it in, the device is identified as a mouse or webcam and the correct device driver is downloaded from the internet and installed. In the time before Plug&Play, each device came with a CD or floppy disk containing the driver. This was always a hassle, Plug&Play has taken away that pain.

The actual model is stored in a global Device Models Repository. You can create your own repository too.

Look at this Azure IoT Plug&Play architecture/flow:

Here are see the different steps needed to build a solution based on Azure IoT Plug&Play:

1. Devices exposing their Model ID
2. IoT Hub storing the Model ID as a reference in the IoT Hub registry
3. Consuming this Model ID by other Azure resources
4. Looking up the actual Model in a Device Models Repository
5. Building up a tree of device capabilities based on the device model

Once the device capabilities are known, an actual UI can be generated for this device so users can interact with it without any extra effort.

Let’s check out how this works.

Cloud IoT dashboards using Grafana with Azure IoT

Azure IoT offers a great solution for connecting IoT devices to the cloud and communicating with them in a secure way and in a two-way fashion: D2C and C2D.

Once you start ingesting telemetry you probably, at some point, want to represent the data in some kind of dashboarding.

This can either be a custom dashboard that gives you the most flexible way to represent the data. I have shown how to do this with Blazor. Or, you could choose PowerBI which is a well-known and productive tool used by many Data Scientists already.

Recently, our team invested some time in building dashboards using Grafana.

With Grafana you can create, explore and share all of your data through beautiful, flexible dashboards.

Azure supports Grafana in various ways in the Azure Marketplace.

For this blog post, I selected the official Grafana template which is hosted in a single VM:

The telemetry is ingested by an IoT Hub and send to a SQL Azure database using Azure Stream Analytics.

As you will see, this is quite an elaborate solution due to all the Azure resources being used.

Still, the solution is quite straightforward and certainly interesting if you are already familiar with Grafana.

Sending telemetry from IoT Edge to IoT Central

After writing my last post, I kept thinking about sending messages to the Azure IoT Central Bridge from Azure IoT Edge.

Eventually, I started building this IoT Edge module which passes incoming route messages to the IoT Central Bridge of your choice.

This module is now available in Docker Hub.

Use it in your IoT device with tag:

svelde/iot-edge-ic-bridge:1.1.0-amd64

Let’s check this module out.

When your clone is several commits behind the master

Once in a while, I clone a project just to check out the code, have it compiled or even to request a pull for an update.

That’s ok.

But what if my clone gets stale? What if it is several commits behind the original repository? You see a message like this:

How do you fix it? Is there a hidden button somewhere in the GitHub portal?

Visualize Azure IoT Edge device routes as a flowchart in Asp.Net MVC

If you look at the routes page in Azure IoT Edge configuration wizard, what do you prefer?

The current notation:

Or do you prefer a flow chart like this:

The routes in Azure IoT edge are a clever solution to describe how messages from one module are sent to another. But the JSON notation can become less readable once you add more (up to twenty) modules. That could end up eg. nineteen routes or more!

Just as an experiment I was thinking about how the ease the experience using a graphical interface.

I prefer the second solution, probably just like you.

So let’s look at how you can create the same experience with your routes of your IoT Edge device.

Create your own local Azure IoT Edge dashboard

Earlier this year, when Azure IoT Edge was still in Public Preview, I wrote a couple of blogs about Visualizing Azure IoT Edge using local dashboard.

Back then, I had to do some magic with both a C# IoT Edge module, a custom NodeJS docker container, and a Docker network to get it running.

Since then, a lot has changed. Microsoft already released a ton of new features. a And there is still more to come regarding the Azure IoT platform.

But that awkward local dashboard solution was nagging me. A few months ago, Microsoft introduced a NodeJS module as a first-class citizen for IoT Edge modules.

So it was time to pick up the gauntlet and use NodeJS for this awesome local IoT Edge dashboard:

#tldr;  If you like to dig into the code, zip it, clone it, extend it or even make a pull request, I made this project open source. If you only want to use it the easy-going way, pull it from docker eg. ‘svelde/localdashboard:1.0.1-amd64’.

At this moment, only Linux containers are supported. It is tested both on Windows and Ubuntu as host OS.

Interested in this module? Let’s see how you can use it.

Cheap Arduino mesh using RF24 radio modules

Communication between devices is key when you want to combine one or more devices. In my previous blog, I used I2C, Bluetooth and RF24 modules. The latter ones, the RF24 have a few advantages: they are cheap and do not need pairing. In my previous blog, I used them for a peer-to-peer (or better: Master-Slave) communication. This time, we will look at a mesh network using these devices.

Getting IP-Address in Win 10 IoT Core

I was working on a new pet project for Windows 10 IoT Core.

At this moment Win10 IoT Core is not supporting the Wi-pi Wi-Fi dongle. So I am using a Dlink 505 mobile companion to attach my Raspberry Pi b2 to my local network. This works great although Visual Studio has some trouble detecting the device.

But because the default app (yes, this is just an app) on the Pi shows the current IP-address, I can simply submit that address when deploying (in the project settings).

So far so good.

But when I deployed my own app I ran into a little problem. My Pi gets the current IP-address dynamically so at one point my deployment was not working anymore. The device had a new IP-address….

And this address is also needed for RDP so it is rather important.

I wanted to read the current IP address in C# code so I could show it in my own app. This seems a bit trivial, but Google was not helping me this time. All examples were based on System.Net so that was not working.

Then I was thinking, could it be that the default app is just open sourced? That way I could look in the code MSFT provided.

And yes it is! Just go to the github location.

And there I found the code to read the IP4 address (now a bit modified):

public static string GetCurrentIpv4Address()
{
  var icp = NetworkInformation.GetInternetConnectionProfile();
  if (icp != null
        && icp.NetworkAdapter != null
        && icp.NetworkAdapter.NetworkAdapterId != null)
  {
    var name = icp.ProfileName;

    var hostnames = NetworkInformation.GetHostNames();

    foreach (var hn in hostnames)
    {
      if (hn.IPInformation != null
          && hn.IPInformation.NetworkAdapter != null
          && hn.IPInformation.NetworkAdapter.NetworkAdapterId
                                                     != null
          && hn.IPInformation.NetworkAdapter.NetworkAdapterId
                      == icp.NetworkAdapter.NetworkAdapterId
          && hn.Type == HostNameType.Ipv4)
      {
        return hn.CanonicalName;
      }
    }
  }

  return "---";
}

Resolve the namespaces if needed…

So thank you MSFT for making the code available.