Connecting child devices to the Azure IoT transparent Edge gateway

Getting started with Azure IoT Edge is easy. Microsoft offers quite some tutorials for several operating systems for setting up an edge gateway.

Once you have created your first IoT edge solution and played with it, you discover Azure IoT Edge takes a bit more time to master.

In real-life IoT is hard, though…

This is because there are more moving parts like security, provisioning, managing, monitoring, etc.

For example, take a look at the ‘iotedge check’ output on your edge device:

This feature of the iotedge runtime makes it possible to check how well your runtime is hardened against common scenarios where something can fail (eg. running out of disk space due to extensive logging or firewall blockades for certain protocols).

In this case, a message is shown which indicates the runtime is using a development (x509) certificate which will expire within ninety days. Communication between the edge modules will stop after that date. A reboot/restart of the runtime is needed to get the runtime running again for another ninety days.

What is the purpose of this certificate and why do we need this to be fixed?

As seen in the documentation:

IoT Edge certificates are used by the modules and downstream IoT devices to verify the identity and legitimacy of the IoT Edge hub runtime module

So, apart from the secure connection with the cloud (either with a symmetric key, x509 certificate, or a TPM endorsement), this certificate is used to secure the communication between modules and possible edge devices. If the certificate expires, edge communication comes to a hold.

Let’s check out how to ruggedize the communication.

Do we need to create multiple certificates?

Yes, it’s not just one certificate. We have to create several certificates related to each other. The main reason is the administration of roles which needs to be kept separate:

  • Edge device manufacturer
  • Edge device operator

If you combine this with the possibility to generate separate certificates for multiple customers and locations eg. factories (if needed) you end up with something like this:

Diagram of typical certificate relationships

Luckily, Microsoft provides good documentation explaining the different certificates and the use cases.

Transparent gateway

This brings us to the main topic of this blog, the transparent gateway:

Microsoft identifies three kinds of IoT Edge gateways, each taking another approach to communicate with other devices:

  1. Protocol translation
  2. Identity translation
  3. Transparent gateway

The first two kinds of gateways are also able to connect to child devices, just like the transparent gateway. Though, with protocol translation, the cloud has registered the gateway only. Child devices do not exist for the IoT Hub registry. The gateway accesses the child devices without any IoT Hub interference.

With identity translation, the children do not know they are part of any IoT solution although there is a registration available for them in the IoT Hub. The gateway does the call to the child devices and wraps the response messages as if these are sent by a child device. The edge holds all the security identities of the child devices.

The transparent gateway is actually ‘transparent’. By default, child devices are capable to connect to the IoT Hub in the cloud. These child devices typically make use of IoT Device SDK features and have eg. a connection string with symmetric key support.

But, through the combination of a reconfigured connection string and making use of an extra certificate (so it can attach to the EdgeHub module routing), the child device telemetry is sent to the transparent gateway. The logic of the Child device is unaware of this all, it thinks it’s still talking against the cloud:

There are multiple advantages to using a transparent gateway:

  • Connection multiplexing – All devices connecting to IoT Hub through an IoT Edge gateway use the same underlying connection. So only one device has to be configured for outbound traffic through the firewall
  • Traffic smoothing – The IoT Edge device will automatically implement exponential backoff if IoT Hub throttles traffic, while persisting the messages locally. This benefit makes your solution resilient to spikes in traffic
  • Offline support – The gateway device stores messages and twin updates that cannot be delivered to IoT Hub
  • Analytics at the edge – Use AI services locally to process data coming from one or more or all downstream devices without sending full-fidelity telemetry to the cloud. Find and react to insights locally and only send a subset of data to IoT Hub

A good starting point for creating certificates for a transparent gateway is seen in this documentation.

If you want to experience the use of a transparent gateway yourself, please check the MS Learn module ‘Set up an IoT Edge Gateway’. This exercise makes use of both the Azure Cloud and your laptop to create an edge gateway in the cloud and a child device on your laptop.

Note: MS Learn is free. That means you can do all kinds of exercises (including Azure portal exercises) without paying for any Azure cloud subscription. All Azure exercises are sandboxed. You just get access to the Azure cloud for a few hours so you can start and finish your exercise while you create and consume actual Azure resources for the duration of the exercise.

Creating certificates for the secure routing

The MS Learn module is a bit awkward though.

Here, an Azure IoT edge device is created in the cloud on a VM and an Azure IoT child device application is running on your laptop. The evil internet is put in between. This means the training includes opening up firewalls and changing hostnames due to the internet in between the two devices.

This is confusing.

In real-life, the architecture is quite the opposite, both edge gateway and child devices are put behind the firewall and share at least one common (local) network.

So let’s create the certificates needed for secure routing on an IoT Edge gateway.

Note: later on, we will attach IoT child devices using the same certificates generated earlier.

First, register an IoT Edge device on the IoT Hub with a few modules:

I added a heartbeat module, just to generate some messages sent to the cloud using the generic routing:

FROM /messages/* INTO $upstream

With the tutorial in mind, I created the needed self-signed certificates using the scripts provided by Microsoft:

git clone https://github.com/Azure/iotedge.git
cd <path>/iotedge/tools/CACertificates
./certGen.sh create_root_and_intermediate
./certGen.sh create_edge_device_identity_certificate "[fill in the devicename of your edge like 'edgedevice']"
./certGen.sh create_edge_device_ca_certificate "[fill in the CA cert name for the same device like 'edgedeviceca']"

Note: It is recommended not to generate all these certificates on the gateway. Keep them in a central secure location.

Note: Check out certGen.sh there you will find out a default password is used to generating the self-signed certificates. The expiration date is set to 30 days (which is even shorter living than the original development certificate). I urge you to change both of them.

Note: In production, the use of self-signed certificates is not recommended. There can be costs involved using a certificate supported by a registar.

From all certificates generated, we need three certificates on our edge device. The names and the related path have to be added in the config.yaml of the edge device:

sudo nano /etc/iotedge/config.yaml

-- Add these lines in the 'certificates' section, save the file and close it.
certificates:
  device_ca_cert: "/home/adminsv/crt/certs/iot-edge-device-ca-uno2372gCA-full-chain.cert.pem"
  device_ca_pk: "/home/adminsv/crt/private/iot-edge-device-ca-uno2372gCA.key.pem"
  trusted_ca_certs: "/home/adminsv/crt/certs/azure-iot-test-only.root.ca.cert.pem"

-- restart the runtime
sudo systemctl restart iotedge
sudo iotedge status iotedge

If all goes well, you will see the arrival of the heartbeat messages in the IoT Hub:

And if you execute the ‘iotedge check’ logic of the iotedge runtime again, you see the ‘production’ certificate is now in place (together with all the other hardening steps to execute):

So now we have a secure and hardened IoT Edge gateway.

Check the edgeAgent logging for the certificate in use:

The beauty is, we have already generated all certificates needed to support the child device connection. With securing the certificate, our gateway is automatically turned into a transparent gateway.

Let’s make use of this feature and connect to a child device right away.

Sending child device messages using C#

If we follow the MS Learn module, we first have to register a child device in the IoT Hub registry. We can then use the generated symmetric key in the C# example code from the MS Learn module. Here a ‘testdevice’ registration is created:

This image also shows the parent-child relationship between the transparent gateway (parent) and C# application (child, testdevice).

Note: A child can only be added to one edge device. Nesting of Edge devices is not supported yet.

The code of the C# app can be found here. Before executing this code, we have to do two things:

  1. Add the hostname to the connection string of the ‘device’
  2. Add the right certificate
  3. Add some identification of messages coming from this device

Let’s dig into these three steps a bit deeper.

Add the hostname

The child device must be able to find the edge gateway within the local network. For this, it needs the hostname of the edge device which it will try to resolve into an IP address.

The hostname of the edge device can be found in the config.yaml. During the installation of the runtime it was added:

On the child device, you can try to resolve it yourself too.

First, ping it the hostname:

If you can ping it, try to resolve it into an IP address:

If this works, add the hostname to the connection string:

private readonly static string connectionString = "HostName=edgedemo-ih.azure-devices.net;DeviceId=testdevice;SharedAccessKey=[symmetric key];GatewayHostName=adminsv-UNO-2372G-J031AE";

Add the right certificate

Next to that, add the ‘azure-iot-test-only.root.ca.cert.pem’ to the application:

This certificate should be generated already.

Add some identification of messages coming from this device

Because the device name is not part of the body of the telemetry message, it is recommended to add something to each message so we can see it’s coming from this child device.

For this we add an extra message property:

message.Properties.Add("temperatureAlert", (currentTemperature > 30) ? "true" : "false");

// Extra message property to identify messages sent by nodejs
message.Properties.Add("source", "csharp");

// Send the telemetry message
await deviceClient.SendEventAsync(message);

Note: we will see the usage of this property later on.

Start the child device

You are now ready to run the child device application. start it with:

dotnet run

The child device will start sending messages to the cloud, through the gateway:

In the IoT Hub, we see the arrival of the TestDevice messages:

Note: The child device messages are part of the local persistent storage of the EdgeHub module. In case the cloud connection is temporarily interrupted, the messages are stored locally. Once the connection is restored, the messages are sent to the cloud after all.

This is great. Though, this does not prove the messages arrive at the transparent gateway…

A simple solution to this problem is revoking the upstream route. By removing upstream communication, you will see a drop in the incoming messages for ‘testdevice’.

A more intelligent way is constructing a special route!

Checking the arrival of child messages at the transparent gateway

At this point, the messages from the child should arrive at the transparent gateway already.

These messages are part of the original routing mechanism which supports messages sent between modules and the edgeHub module (upstream).

We can identify the child messages using this special route:

FROM /messages/* WHERE NOT IS_DEFINED($connectionModuleId) INTO BrokeredEndpoint("/modules/echo/inputs/input1")

Child device messages lack a connection module id. Sure these are not generated by some module…

This route looks like:

Note: The echo module is just a handy debug module to visualize messages (including message properties) in the console.

Once the route is in place, we can check for incoming messages using the echo console:

As you can see, the messages are sent to the transparent gateway. The messages have that distinctive message property!

Note: the device name was not part of the message body. Using that extra ‘source’ property, we are still able to separate messages coming from multiple child devices.

On top of that, you can route them to any module you want. So you can combine information from multiple child devices for better insights and actions.

Sending child device messages using NodeJS

Speaking about other child devices, let’s add a few!

First, we add a NodeJS child device using example code found here.

The resulting code looks like this after we added extra lines of code so it can act like a child device:

// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

'use strict';

// The device connection string to authenticate the device with your IoT hub.
//
// NOTE:
// For simplicity, this sample sets the connection string in code.
// In a production environment, the recommended approach is to use
// an environment variable to make it available to your application
// or use an HSM or an x509 certificate.
// https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-security
//
// Using the Azure CLI:
// az iot hub device-identity show-connection-string --hub-name {YourIoTHubName} --device-id MyNodeDevice --output table
var connectionString = 'HostName=edgedemo-ih.azure-devices.net;DeviceId=nodejs;SharedAccessKey=[primary SAS key];GatewayHostName=adminsv-UNO-2372G-J031AE';

// Using the Node.js Device SDK for IoT Hub:
//   https://github.com/Azure/azure-iot-sdk-node
// The sample connects to a device-specific MQTT endpoint on your IoT Hub.
var Mqtt = require('azure-iot-device-mqtt').Mqtt;
var DeviceClient = require('azure-iot-device').Client
var Message = require('azure-iot-device').Message;

var client = DeviceClient.fromConnectionString(connectionString, Mqtt);

// Extra child device code required to connect the transparent gateway edgehub using CA certificate 
var options = {};
options.ca = 'azure-iot-test-only.root.ca.cert.pem';
process.env.NODE_EXTRA_CA_CERTS = 'azure-iot-test-only.root.ca.cert.pem';
client.setOptions(options);

// Create a message and send it to the IoT hub every second
setInterval(function(){
  // Simulate telemetry.
  var temperature = 20 + (Math.random() * 15);
  var message = new Message(JSON.stringify({
    temperature: temperature,
    humidity: 60 + (Math.random() * 20)
  }));

  // Add a custom application property to the message.
  // An IoT hub can filter on these properties without access to the message body.
  message.properties.add('temperatureAlert', (temperature > 30) ? 'true' : 'false');
  
  // Extra message property to identify messages sent by nodejs
  message.properties.add('source', 'nodejs');

  console.log('Sending message: ' + message.getData());

  // Send the message.
  client.sendEvent(message, function (err) {
    if (err) {
      console.error('send error: ' + err.toString());
    } else {
      console.log('message sent');
    }
  });
}, 1000);

Three additions are made:

  1. The connection string is extended with the GatewayHostName
  2. Extra lines for attaching the same ‘azure-iot-test-only.root.ca.cert.pem’ certificate as seen in the C# example
  3. The message has gotten an extra message property named ‘source’ to identify messages arriving from this NodeJS child device at the transparent gateway

This code can be started using NodeJS:

node .\SimulatedDevice.js

Of course, to get the symmetric key secrets, we first needed to register a child device in the IoT Hub and attach a parent to it:

Note: As you can see, I also added a NodeRed child device.

Bonus: Node Red as a child device

Recently, Microsoft released a NodeRed node as Azure IoT Device. This node has many features, including support for being a child device.

Note: yes, this is the upgraded node from my previous blog:

We need to register a child device for this node too:

And, again, we need to attach the same hostname and certificate as used within the other devices too:

Note: I got it working on a Raspberry Pi using the Azure IoT Edge NodeRed module. I used MQTT as the protocol between child device (so in fact it is another Azure IoT Edge device that can be kept unaware of the child device situation inside the NodeRed module). At first, I got some connection errors on the Azure IoT device node. I had to reboot the device to get it working. I’ve been informed this is due to some issues with MQTT inside NodeJS.

Note: Possibly just restarting the NodeRed module could do the trick too.

Now, here we see the incoming messages from all three child devices:

As you can see, when this picture was taken, no message properties were supported by the node.

Starting 2020-10-12, the NodeRed node is now supporting message properties, after discussing this with the owner of the Azure IoT Device node:

Conclusion

Here we see a fine example of why Azure IoT child devices are a great addition to the already excellent stack of Azure IoT Features.

We are now able to mix and match messages coming from multiple child devices, all being unaware of the transparent gateway.

We are even able to already connect multiple IoT Edge gateways using NodeRed!

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen.

WordPress.com logo

Je reageert onder je WordPress.com account. Log uit /  Bijwerken )

Google photo

Je reageert onder je Google account. Log uit /  Bijwerken )

Twitter-afbeelding

Je reageert onder je Twitter account. Log uit /  Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit /  Bijwerken )

Verbinden met %s