Getting started with Azure IoT Plug and Play

Microsoft is the founder of this concept called Azure IoT Plug and Play:

IoT Plug and Play enables solution builders to integrate smart devices with their solutions without any manual configuration.

The idea is that the device describes itself using some identification key. This key, the Model ID (or DTMI, Device Twin Model Identification), is bound to a complete model written in DTDL (Digital Twins Definition Language).

Using this model, the interface (or capabilities) of this device can be read:

  • Properties (Azure IoT Device Twin desired properties and reported properties)
  • Telemetry (the D2C messages)
  • Command methods (based on Azure IoT Direct methods)

Once a device starts communicating using this deterministic interface, a User Interface can be provided dynamically.

This is the same principle of Plug and Play devices like a mouse or a webcam. If you plug it in, the device is identified as a mouse or webcam and the correct device driver is downloaded from the internet and installed. In the time before Plug&Play, each device came with a CD or floppy disk containing the driver. This was always a hassle, Plug&Play has taken away that pain.

The actual model is stored in a global Device Models Repository. You can create your own repository too.

Look at this Azure IoT Plug&Play architecture/flow:

Here are see the different steps needed to build a solution based on Azure IoT Plug&Play:

  1. Devices exposing their Model ID
  2. IoT Hub storing the Model ID as a reference in the IoT Hub registry
  3. Consuming this Model ID by other Azure resources
  4. Looking up the actual Model in a Device Models Repository
  5. Building up a tree of device capabilities based on the device model

Once the device capabilities are known, an actual UI can be generated for this device so users can interact with it without any extra effort.

Let’s check out how this works.

Note: All the code used in the blog post is open source and available at GitHub. This GitHub location also acts like a custom Device Models Repository location.

We need a DTDL model and a DTMI

I want to demonstrate how it works in this blog post, therefor we see the usage of two DTMI Model IDs:

  1. dtmi:com:example:NotExisting;1 – This is just a unique Model ID (I made it up myself). There is no actual Model created. We need this just to get started
  2. dtmi:com:example:TemperatureController;1 – A Model ID provided by Microsoft, including a DTDL model. It is used by Microsoft in many PnP examples. We reuse it later on within our own repository

I use the non-existing Model ID to prove I can register it into the IoT Hub.

I used the Microsoft Model and Model ID together to prove I can read the model interface from any DTDL Model catalog repository, including my own GitHub repository.

Exposing the DTMI by the device client

Microsoft provides SDKs in many languages for building Azure IoT devices. These SDKs make it very simple to connect a device to the Azure IoT Hub. It is also the preferred way to expose the Model ID to the IoT Hub.

A simple device Model ID example

Once a device connects, the Model ID is added to the Azure IoT device Device Twin under ‘modelId’:

This Model ID was automatically added by this code on connecting with the IoT Hub (using the optional ClientOptions class):

internal class Program
{
    private static void Main(string[] args)
    {
        var connectionString = "HostName=edgedemo-ih.azure-devices.net;DeviceId=pnpSimpleDevice;SharedAccessKey=[SECRET]";
        using var deviceClient = DeviceClient.CreateFromConnectionString(
            connectionString,
            new ClientOptions { ModelId = "dtmi:com:example:NotExisting;1" });
        deviceClient.OpenAsync().Wait();

        SendSingleMessage(deviceClient);
    }

    private static void SendSingleMessage(DeviceClient deviceClient)
    {
        string jsonData = "{ \"meaning\":42}";
        using var message = new Message(Encoding.UTF8.GetBytes(jsonData));
        message.Properties.Add("messagetype", "normal");
        deviceClient.SendEventAsync(message).Wait();
        Console.WriteLine("A single message is sent");
    }
}

So, all it takes is exposing that DTMI using the Device Client connection.

The initial device registration in the IoT Hub had no model put in the Device Twin. Only after the device connected, that Model ID was added.

Note: as you can see there is no relation between the Model ID and the actual messages sent by the device to the cloud. The model is just a convention. You are responsible to follow your own convention!

The code for this example is available here.

A less simple device example using Azure Device Provisioning Service

Although this second example is a bit more elaborate (it first enrolls the device using a DPS and only then connects to the IoT Hub), the outcome is the same:

Here, the Device client is created using the IoT Hub device security credentials retrieved after the enrollment.

When the connection is made by the Device client, the Model ID is inserted in its Device Twin:

...
IAuthenticationMethod auth = new DeviceAuthenticationWithRegistrySymmetricKey(
    result.DeviceId,
    security.GetPrimaryKey());

using DeviceClient iotClient = DeviceClient.Create(
    result.AssignedHub,
    auth,
    new ClientOptions { ModelId = "dtmi:com:example:NotExisting;2" });

string jsonData = "{ \"meaning\":43}";
using var message = new Message(Encoding.UTF8.GetBytes(jsonData));
await iotClient.SendEventAsync(message);
...

This results in this Model ID registration in the Device Twin:

This means we are able to expose the Model ID to the IoT Hub, even when the DPS is part of the solution.

It’s up to you to keep the actual device interface capabilities on par with the DTML model behind the Model ID.

The source code of this example can be found here.

Exposing the Model ID of the IoT devices

Image devices connecting to the IoT Hub and starting to send their messages.

This is a nice opportunity for building a UI dashboard capable of listing the available devices, each with the Model ID. And from there, building up a dynamic UI based on the capabilities in the underlying model.

Unfortunately, the Model ID is not added to the incoming messages (not even using message enrichment).

So, when the first messages come in, your dashboard is still clueless regarding the model interfaces the messages are based on.

There are three ways to get access to a device Model ID:

  1. The IoT Hub exposed changes in Device Twins (actually, Digital Twin Change events)
  2. You can use the Azure IoT Server SDK to read (RegistryManager class)
  3. using the Azure IoT Hub Rest API (out-of-scope in this blog post)

Let’s check out the first two ways.

Getting your hands on the Model ID using the IoT Hub event Route

Here, I created this custom route in the Azure IoT Hub message routing mechanism.

So we are going to see a ‘Message Push’ implementation.

This route listens to any Digital Twin Change Events:

Note: Next to this data source, there is also a Device Twin Change Events data source but that one does not expose the Model ID.

The IoT Hub message routing mechanism uses an endpoint registration to send specific messages to specific endpoints (blob storage, event hub, message bus).

Here, I send messages representing Azure Digital Twin changes to an Azure Blob Storage container:

Once the route is active and a device updates its ModelID, we see this message appear when that ADT change is detected:

Yes, the ‘digitalTwinChangeNotification’ message is sent due to a twin update:

{
    "EnqueuedTimeUtc": "2021-07-18T13:00:39.6860000Z",
    "Properties": {
        "hubName": "edgedemo-ih",
        "deviceId": "PnPDeviceViaDps",
        "operationTimestamp": "2021-07-18T13:00:39.6825493Z",
        "iothub-message-schema": "digitalTwinChangeNotification",
        "opType": "updateTwin"
    },
    "SystemProperties": {
        "correlationId": "14b74261c1ce",
        "connectionDeviceId": "PnPDeviceViaDps",
        "contentType": "application/json-patch+json",
        "contentEncoding": "utf-8",
        "userId": "ZWRnZWRlbW8taWg=",
        "enqueuedTime": "2021-07-18T13:00:39.6860000Z"
    },
    "Body": [
        {
            "op": "replace",
            "path": "/$metadata/$model",
            "value": "dtmi:com:example:NotExisting;2"
        }
    ]
}

Notice that the reason for this message being sent is the replacement of the Model ID.

So, we are notified when a device exposes a Model ID as a ‘push event’.

If a more elaborate solution, the message should be exposed by an event mechanism like an Event Hub. This message is then picked up by eg. an Azure function that executes custom code.

Getting your hands on the Model ID using the IoT Hub Server SDK

If you have no time to wait until these messages arrive, you can always make a call towards the IoT Hub to see if a device has a Model ID. You ‘pull’ the Model ID.

Again, Microsoft provides Server SDKs in many languages supporting programming against the IoT Hub.

The call to make is very straight-forward using the RegistryManager:

using Microsoft.Azure.Devices;
using System;
using System.Threading.Tasks;

namespace RetrieveModelIdentificationApp
{
    internal class Program
    {
        private static async Task Main(string[] args)
        {
            var manager = RegistryManager.CreateFromConnectionString("[IOT HUB Connectionstring]");
            var twin = await manager.GetTwinAsync("PnPDeviceViaDps"); 
            var modelId = twin.ModelId;

            Console.WriteLine($"The current model is {modelId}");
        }
    }
}

All you need is to reference the right NuGet package:

<PackageReference Include="Microsoft.Azure.Devices" Version="1.34.0" />

Once the IoT Hub connection string is entered, the Model ID is exposed:

As you can see this is a ‘pull’ instead of a ‘push’.

The code for this example is seen here.

Bonus: Azure IoT (Plug and Play) Explorer

The Azure IoT Explorer can also be used to explore IoT Devices. It exposes the Model ID too:

I tested this with another IoT device exposing that Microsoft sample DTMI:

As you can see, a model definition with all capabilities is retrieved based on the Model ID.

Now the UI can be generated related to eg. the direct methods to manipulate and execute these methods.

For example, a ‘reboot’ command can be executed:

This is great.

We now go back to our Model ID, this is still the only thing we received right now.

What’s next?

Device Models Repositories

Okee, so we know which device has which Model ID.

But this is not enough to build that dynamic UI on, we need to get access to a Device Models Repository where all the descriptions of the capabilities of a device can be found.

We need the Device Model!

Microsoft provides an official Device Models repository: https://devicemodels.azure.com/

If you go there using a browser, you get this message:

The idea is that you programmatically find the DTDL model there based on the Model ID.

If you look twice, you see it’s based on a Github repo:

Actually, the Model ID, the DTMI, is just a description of the path within this repository!

For example, look at this Microsoft DTMI:

dtmi:com:example:TemperatureController;1

It is just this GitHub path:

Note: Microsoft also provides a nice wrapper around this GitHub repo. This model is also available at https://devicemodels.azure.com/dtmi/azure/devicemanagement/deviceinformation-1.json.

Note: I have not investigated research time in this REST interface. As you see below, GitHub automatically provides a working REST API for me 🙂

So we have to program custom code?

No problem.

Microsoft provides NuGet libraries to interact with these repositories, even via this GitHub repository:

<PackageReference Include="Azure.IoT.ModelsRepository" Version="1.0.0-preview.3" />
<PackageReference Include="Microsoft.Azure.DigitalTwins.Parser" Version="3.12.5" />

As we will see in a moment, the tooling also supports custom repositories, next to the official one.

To demonstrate this, I actually built one using my own GitHub account!

I just took the example model from Microsoft and put it in my own GitHub repository:

To do this, you need to do two things in GitHub:

  1. Make your repository public, not private
  2. Know the name of your branch (Microsoft uses ‘main’, mine is still ‘master’)

With that, we are ready to upload models.

Looking up the actual Model in a Device Models Repository

At this point, we have both a ModelID and a (custom GitHub) repository path.

My custom path is

https://raw.githubusercontent.com/sandervandevelde/DTMI-Starter/master

This is enough to make a call:

...
// note: 'master' is the publicly available branch
var uri = new Uri("https://raw.githubusercontent.com/sandervandevelde/DTMI-Starter/master");

var dtmi = "dtmi:com:example:TemperatureController;1";
var isValidDtmi = DtmiConventions.IsValidDtmi(dtmi);
Console.WriteLine($"isValidDtmi: {isValidDtmi}");

var modelUri = DtmiConventions.GetModelUri(dtmi, uri, false);
var clientRaw = new ModelsRepositoryClient(
        uri,
        new ModelsRepositoryClientOptions(dependencyResolution: ModelDependencyResolution.Enabled));
IDictionary<string, string> models = clientRaw.GetModels(dtmi);

var parser = new ModelParser();
IReadOnlyDictionary<Dtmi, DTEntityInfo> parseResult = await parser.ParseAsync(models.Values.ToArray());

Console.WriteLine($"The following {models.Count} interfaces are resolved with {parseResult.Count} entities.");

models.Keys.ToList().ForEach(k => Console.WriteLine($"Model: {k}"));
parseResult.Values.ToList().ForEach(v => Console.WriteLine($"element: {v.GetType()}"));
...

Note: The GitHub repo is called using an alternative ‘REST’ API https://raw.githubusercontent.com/sandervandevelde/DTMI-Starter/master that GitHub supplies next to the original repository path.

Executing this code results in:

And this is indeed the right model information.

The model is originally a combination of three different (sub) models. Together, these models expose 38 entities (properties, telemetry fields, units, commands, etc.).

Now, it’s up to you to build a UI based on these values. Microsoft is able to do this (eg. in the Azure IoT Explorer and Azure IoT Central). So can you.

Here is an example of how to get your hands on the telemetry details:

foreach (var p in parseResult)
{
    if (p.Value is DTTelemetryInfo)
    {
        Console.WriteLine($"parse '{p.Value}': '{p.Key}' ");
        Console.WriteLine($"  Id '{p.Value.Id.AbsolutePath}' ");
        Console.WriteLine($"  DisplayName '{p.Value.DisplayName.First().Key}' - '{p.Value.DisplayName.First().Value}'");
        Console.WriteLine($"  EntityKind {p.Value.EntityKind}");
        Console.WriteLine($"  Description {p.Value.Description.First()}");
        Console.WriteLine($"  DefinedIn {p.Value.DefinedIn.AbsolutePath}");
        Console.WriteLine($"  Comment {p.Value.Comment?.First()}");

        var pt = p.Value as DTTelemetryInfo;
        Console.WriteLine($"  pt.EntityKind = {pt.EntityKind}");

        if (pt.Schema is DTDoubleInfo)
        {
            var ptdouble = pt.Schema as DTDoubleInfo;
            Console.WriteLine($"  ptdouble {ptdouble.EntityKind} - {ptdouble.Description.First().Value} ");
        }
    }
}

This results in this telemetry information:

Here, we see eg. a temperature field as part of the telemetry which is a ‘double’. So, a simple gauge or line chart could be used to represent this incoming telemetry data.

The code used in this example is available here.

Bonus: Custom Device Models Repository support in Azure IoT Explorer

The IoT Explorer supports custom Device Model repositories too:

It seems to need the REST version of a custom repository, so I removed the public repository and added my own (configurable) repository as the single repository:

When I try to open the IoT Plug and Play components page for my device exposing that Microsoft DTMI, it succeeds:

It even tells me the model is indeed resolved from my own Configurable repository!

The source code of the IoT Explorer is open source, perhaps this is a good starting point for you if you want to build a configurable UI?

Conclusion

These are all the tools you need to build your own Azure IoT Plug and Play UI.

The REST API provided by GitHub is just perfect for your own custom Device Models Repository.

Azure IoT Explorer is a good tool for testing your device model.

Once you have a solid set of models, you could start certifying them so they can appear on the public repository. These devices are already certified by Microsoft, check out that catalog. The IoT Plug and Play certification requirements are available here.

Then, these modules are also available for Azure IoT Central.

Azure IoT Edge modules can also expose a Model ID using their Module Twin.

Advertentie