How to create Azure IoT Module Identities via DPS

For those working with Azure IoT Edge, the support for Azure IoT Edge modules is a well-known mechanism to deploy pieces of IoT logic to (edge) devices.

Every Azure IoT Edge module functions as a Docker container with an independent message flow, module twin, tags, desired properties, reported properties, and direct methods.

The modules are bound to the Azure IoT Edge device and rely on the security and communication channel provided by that device registration.

Azure IoT also supports the concept of Module Identities.

This ability is less known and used, which is a pity if you consider all the options it brings…

Module Identities are registered as part of any Azure IoT device. This can be an edge device but it also works for ‘generic’ Azure IoT devices.

Module Identities have their own credentials, something an Azure IoT Edge module does not have.

This way, Module Identities can send their own messages, have a module twin, and support direct methods.

Why should we use Module Identities?

If you consider your devices as being modular by design, with different parts of data collection, perhaps even by different manufacturers, you can separate concerns and security by giving each part its own module identity. It even fits well into Azure IoT Plug&Play.

Unfortunately, the Device Provisioning Service does not support Module Identities, only Device Identities. I expect this is why Module Identities never became as popular as it should be.

But this is not the end of the story…

Let’s see how we can work around this shortcoming.

Disclaimer: I demonstrate a workaround with the latest V2 SDKs for both the Azure IoT Hub Service and the Azure IoT Devices. These are still in beta so things could change in the future. I’m unaware if this works with any other SDK written in eg. C# V1 SDK, Python, C, Java, or Javascript also. Because it’s based on the same IoT Hub service REST API, it probably will…

Note to my future self: Please check out if this is still working when the V2 SDKs are GA.

Microsoft uses these Module Identities too

Not only customers use Module Identities to separate concerns.

Microsoft also advertises multiple tools where the security is based on the same Module Identities:

• Azure Defender For IoT agent.
• Device update for Azure IoT Hub.

So, Module Identities are used to secure local logic, sometimes running next to Azure IoT Edge.

As you can see in the documentation, access to one or more Module Identity connection strings is needed so we can use them for these tools.

This implies that we have to create upfront both the device on an IoT Hub and the related Module Identities. Only then we can use them for the tools.

This means also that the flexibility the DPS offers regarding creating IoT Hub devices on the fly has no meaning anymore.

Let’s see how we can overcome this limitation.

The normal way of creating a Module Identity

If we check the documentation, we can only create Module Identities using the Azure IoT Hub Service SDK.

Let’s demonstrate this with the latest preview version:

<PackageReference Include="Microsoft.Azure.Devices" Version="2.0.0-preview005" />

We can execute something like this:

Console.WriteLine("Create a Module Identity via IoT Hub Owner connectionstring");

Console.WriteLine("Do not expose this string outside the cloud!");

var clientIoTHubOwner = 
    new IotHubServiceClient(
        "HostName=edgedemo-ih.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=[secret]");

var moduleViaIoTHubOwner = 
    clientIoTHubOwner.Modules.CreateAsync(
        new Module(
            "servicesdkbeta2device", 
            "moduleviaIHOwner")).Result;

Console.WriteLine($"Via IoT Hub owner connectionstring: {moduleViaIoTHubOwner.Id} on {moduleViaIoTHubOwner.DeviceId} with key '{moduleViaIoTHubOwner.Authentication.SymmetricKey.PrimaryKey}'");

A call is made using the IoT Hub IoTHubOwner access policy connection string.

WARNING: Do not attempt to execute this code with the IoT Hub owner connection string on a device! The IoT Hub Owner connection string is not to be used outside Azure. Keep it under lock and key because this gives unlimited power to hackers if they can get to this key. Think about changing routing, sending messages, and removing devices and settings.

We see the Module Identity is created on the device:

In the Azure portal, we can access the Module Identity:

This is an OK solution if you can access the physical device after the Module Identities are created so you can distribute the secrets.

You also miss out on the DPS advantages though.

On the other hand, the Azure IoT device SDK does not offer this functionality.

So how can we create Module Identities on devices?

Creating a Module Identity on the device without an Azure IoT Hub connection string

When I was researching the ability to combine the Device Provisioning Service with Module Identities, I got in contact with the product group in charge.

They hinted to me about a hidden IoT Hub Service SDK feature:

The Azure IoT Hub Service SDK accepts a device connection string too

So I tried this out, on the device, using the device connection string:

Console.WriteLine("Create a Module Identity via a device connectionstring");

Console.WriteLine("This is nice for development an testing!");

Console.WriteLine("It's recommended to work with a DPS instead!");

var clientOnDevice = 
    new IotHubServiceClient("HostName=edgedemo-ih.azure-devices.net;DeviceId=devicesdkbeta2device;SharedAccessKey=[device connection string]");

var moduleViaDeviceCs = 
    clientOnDevice.Modules.CreateAsync(
        new Module(
            "devicesdkbeta2device", 
            "moduleviadevicecs")).Result;

Console.WriteLine($"Via device connectionstring: {moduleViaDeviceCs.Id} on {moduleViaDeviceCs.DeviceId} with key '{moduleViaDeviceCs.Authentication.SymmetricKey.PrimaryKey}'");

When we run this, indeed we create a Module Identity on Azure, in the IoT Hub, related to our chosen device!

We can see the Module Identity in the portal:

We also get access to the primary key (and secondary key) of the Module Identity:

This is perfect!

I agree, it feels a bit like a hack: using the IoT Hub service SDK on a device but providing the device connection string.

But it’s actually a hidden feature used by Microsoft itself already!

Still, it works and this cannot do much harm. Everything the device can do on the IoT Hub is in the context of that same device.

There is only one flaw…

We still need a device connection string.

This means the device must exist already?

Therefore, the advantage of using a DPS is non-existing?

What if we can extend our solution with DPS usage.

Using the Device Provisioning Service to create/access the device

Normally, devices are first registered in a Device Provisioning Service on the first call coming from the IoT device.

Note: I demonstrate this with an individual enrollment of a device with a symmetric key. Technically, this could work for any type of device enrollment.

So, we need:

• The global URI of the Azure Device Provisioning Service platform
• The IDScope of our own DPS
• The DPS device registration name
• The SAS key related to the device DPS enrollment

We want to turn these configuration settings into a new (or already existing) Module Identity with a certain name (like ‘DefenderIotMicroAgent’ in the case of the Defender for IoT agent).

So, we need also:

• Module Identity name

Let’s start.

First, we make use of another DPS Nuget Package.

This is the DPS Service client:

<PackageReference Include="Microsoft.Azure.Devices.Provisioning.Client" Version="2.0.0-preview005" />

This results in this C# static function:

private static (string assignedHub, Module module) ProvideIoTHubDeviceModuleIdentity(string dpsGlobalUri, string idScope, string deviceRegistrationName, string deviceRegistrationSymmetricKey, string moduleId)
{
    var aps =
        new AuthenticationProviderSymmetricKey(
            deviceRegistrationName,
            deviceRegistrationSymmetricKey,
            deviceRegistrationSymmetricKey);

    ProvisioningDeviceClient dpsClient =
        new ProvisioningDeviceClient(
            dpsGlobalUri,
            idScope,
            aps);

    var dpsDevice = dpsClient.RegisterAsync().Result;

    Console.WriteLine($"Registered device: {dpsDevice.DeviceId} at {dpsDevice.AssignedHub} ");

    var cs = $"HostName={dpsDevice.AssignedHub};DeviceId={dpsDevice.DeviceId};SharedAccessKey={aps.PrimaryKey}";

    var ServiceClient = new IotHubServiceClient(cs);

    Module module = null;
    string source = string.Empty;
    try
    {
        module = ServiceClient.Modules.GetAsync(dpsDevice.DeviceId, moduleId).Result;

        source = "Existing module";
    }
    catch (Exception)
    {
        // module does not exist; 
    }

    if (module == null)
    {
        module =
            ServiceClient.Modules.CreateAsync(
                new Module(
                    dpsDevice.DeviceId,
                    moduleId)).Result;

        source = "Created module";
    }

    Console.WriteLine($"{source}: {module.Id} on {module.DeviceId} with key '{module.Authentication.SymmetricKey.PrimaryKey}'");

    return (dpsDevice.AssignedHub, module);
}

This code first connects to the DPS to get access to the device registration.

This forces the DPS to collect information from either the existing device on the linked IoT Hub or the created device in the IoT Hub of choice.

Note: The code asks us to provide both a primary key and a secondary key. Normally, IoT devices only have one key available. I expect the SDK needs a bit more TLC to cope with this. So I cheated a bit, I just provided the same key twice. This works 🙂 In the end, a device will be created with the same primary key but with another valid secondary key.

Then, we construct a device connection string with the device information provided: the IoT Hub device name of the existing device, the name of the linked IoT hub, and the primary key.

Note: the DPS symmetric primary key is reused while creating the IoT Hub device credentials.

Finally, the function returns both the IoT Hub full name the device is assigned to and the complete Module Identity.

We need these later on.

We need another NuGet package so we can access the ModuleClient for sending messages:

<PackageReference Include="Microsoft.Azure.Devices.Client" Version="2.0.0-preview005" />

Making the call is easy:

Console.WriteLine("DPS -> module");

var dpsGlobalUri = "global.azure-devices-provisioning.net";
var idScope = "0ne0XYZABC";
var deviceRegistrationName = "sdk2deviceregistration";
var deviceRegistrationSymmetricKey = "[symmetric primary key]";
var moduleId = "modViaDps";

var result = ProvideIoTHubDeviceModuleIdentity(dpsGlobalUri, idScope, deviceRegistrationName, deviceRegistrationSymmetricKey, moduleId);

Console.WriteLine("Sending a telemetry message...");

var moduleIdentityConnectionstring = $"HostName={result.assignedHub};DeviceId={result.module.DeviceId};ModuleId={result.module.Id};SharedAccessKey={result.module.Authentication.SymmetricKey.PrimaryKey}";

var moduleClient = new IotHubModuleClient(moduleIdentityConnectionstring);

moduleClient.OpenAsync().Wait();

var testMessage = new TestMessage 
{ 
   Created = DateTime.UtcNow, 
   Message = "Test message for module" 
};

var jsonString = JsonSerializer.Serialize(testMessage);

var message = new TelemetryMessage(Encoding.UTF8.GetBytes(jsonString));

moduleClient.SendTelemetryAsync(message).Wait();

Console.WriteLine("telemetry message sent");

moduleClient.CloseAsync().Wait();

First, this results in the creation of this Module Identity:

Then, that information from the IoT Hub full hostname and the Module Identity is used to construct the Module Identity connection string.

Only then we can create an IoTHubModuleClient (for sending messages, receiving direct method calls, etc.).

Once that Module Identity client is constructed, sending a message is easy:

Notice both the device id and module id are part of the device-to-cloud message system properties.

Note: I expect this workaround even works when the symmetric keys are rolled (switching over from primary to secondary key).

If you want to use this together with the Defender for IoT agent or Device Update agent, you need to write some extra logic that creates/reads the Module Identities used by these services. You probably need to restart the services too.

Hopefully, Microsoft will update the agents using this new solution too.

It seems the Microsoft OSConfig agent already uses this feature partially. It creates a module called ‘osconfig’. The demonstrated installation only needs a device connection string. You can find the support for the Device Provisioning Service in the ‘/etc/aziot/config.toml’ file.

Conclusion

You have seen that the Azure IoT Hub service SDK works using a device connection string too.

This is great because we never want to expose any IoT Hub connection string outside the cloud!

Now, we can create new Module Identities inside the device of our choice at our own pace.

Using this workaround, we finally can combine Module Identities together with Device Provisioning Service calls.

This way, we do not have to create devices in advance, roll out IoT Hub connection strings to devices, or create some fancy edge/cloud logic to distribute Module Identity connection strings.

The only drawback is that the same device credentials, together with a unique Module Identity ID, must be shared between multiple Client applications based on Module Identity security.

I hope you understand the immense (DPS) flexibility and increased security outweigh this limitation.

This feature is offered at least by the latest C# SDKs. Please let me know if this works for you too.

I would like to express my gratitude to the Azure IoT product team for guiding me in the right direction!