Message enrichment for Azure IoT Edge modules

The Azure IoT Hub offers ingestion of D2C and D2C messages for thousands or more devices.

Incoming messages are routed to several endpoints (eg. Event Hub or Blob Storage) using internal IoT Hub routes:

Note: even messages sent to the Event Grid are also enriched.

The messages produced by devices look like these (image taken from the console output of the Microsoft temperature simulation):

Here, messages from the Microsoft temperature simulation module are sent to the cloud.

When these messages arrive in the IoT Hub, the fulll message format looks like this:

{
  "body": {
    "machine": {
      "temperature": 73.05171716227275,
      "pressure": 6.929942461524743
    },
    "ambient": {
      "temperature": 20.795734739068774,
      "humidity": 26
    },
    "timeCreated": "2021-05-10T11:51:42.1835163Z"
  },
  "enqueuedTime": "Mon May 10 2021 13:51:42 GMT+0200 (Central European Summer Time)",
  "properties": {
    "sequenceNumber": "101",
    "batchId": "7f3f847f-c20c-49aa-af17-a22db31f244f"
  },
  "systemProperties": {
    "iothub-connection-device-id": "simulation",
    "iothub-connection-module-id": "sim",
    "iothub-connection-auth-method": "{\"scope\":\"module\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}",
    "iothub-connection-auth-generation-id": "637540164221770949",
    "iothub-enqueuedtime": 1620647502173,
    "iothub-message-source": "Telemetry",
    "x-opt-sequence-number": 13975,
    "x-opt-offset": "12885042704",
    "x-opt-enqueued-time": 1620647502314
  }
}

As you can see, the messages are build up in three parts:

  1. The actual message body, generated by (your) logic in the device. This part could be encoded so only the receiver knows how to handle this. Normally this is a JSON message format, though.
  2. (Application) properties, message enrichment on the device to provide context about the messages. This acts as a property bag, also added by the developer. The values are directly accessible for routing purposes.
  3. System properties, added by the IoT Hub. This gives technical context about the incoming message from the perspective of the IoT Hub

So users can choose to add values in the body or to add values in the application properties.

Still, every byte put in the message sent by the device results in a longer transmit duration and extra costs (number of bytes communicated, potentially more message chunks handled by the IoT Hub).

So, values already found in the system properties (eg. the device name or module id) can be left out in the body. This results in a smaller message.

Microsoft also provides message enrichment on the IoT Hub. This can take away some of the pain too.

It also makes it possible to route messages further down the road in other Azure resources which receive these enriched messages.

Let’s check this out.

Doorgaan met het lezen van “Message enrichment for Azure IoT Edge modules”

Adding local persisted state to IoT Edge modules using Module twin

Each Azure IoT Edge module, deployed to a device, has its own Module twin.

A Module twin is the same concept as a Device twin for an Azure IoT Device, it stores state information including metadata, configurations, and conditions.

A Module twin is essentially a JSON document which lives both in the cloud (in the IoT Hub) and on the device and is kept in sync when communication between device and cloud is possible:

In the IoT Hub, the tags are writable and readable. These can be used to identify a specific device with an alternative key and/or to filter subsets of devices.

Also in the cloud, the desired properties can be written with updated values. These (updated) values (eg. properties or settings) are picked up by the device when it is connected. So it could take days or weeks for the updated desired property to be picked up because the device is offline in the meanwhile.

But the desired properties are patient…

Once the updated values of changed desired properties are arriving at a device, a notification method on the device is triggered to handle them.

As a good citizen, an IoT Edge module should report back to the cloud how it is updated by the desired properties. This is done using the reported properties in the Module twin.

This closes the loop for the administrator. I can publish a desired property change for one or more devices. And after a while, the reported properties can be checked to see which devices have picked them up and which devices need some attention.

Do you notice that it’s also possible to read reported properties, on the module side?

Write data, read data… that is enough to persist data on the edge, isn’t it?

Let’s see how we can use this for persisting local state.

Doorgaan met het lezen van “Adding local persisted state to IoT Edge modules using Module twin”