Introduction to the IoT Edge SDK V1, part 1: the modules

Update: Recently, Microsoft introduced the new V2 version of their IoT Edge. This blog is referencing the former but still supported V1.

When the Azure IoT Platform is referenced, in most cases the devices connecting to the IoT Hub are capable of communicating directly with the internet using Wifi etc. But there are many cases where devices are not capable of reaching out to an IoT Hub.

For example, these devices lack the ability to communicate using the internet (but use eg. Bluetooth or I2C instead). Or these devices are capable of communicating eg. REST but simply disconnected from the internet. Or they can only reach their own platform (eg. LORA).

In these cases, you need a mediator, a gateway. It sits between the two parties and passes data back and forth.

Microsoft provides for these cases the IoT Edge SDK, formally known as the Azure IoT Gateway SDK.

This SDK makes it possible to run a service which makes it possible to connect devices to the Azure IoT Hub using a series of modules.

But the name change (from ‘gateway‘ to ‘iot edge’) is not without reason. The Edge SDK has extended logic and is currently in preview. The additions to come will make it possible to run logic on-premise (according to the website: Enable real-time decisions, Perform edge analytics, Run artificial intelligence at the edge, etc.). This is promising!

But I have experienced the usage of the Gateway SDK as a challenge. The SDK supports many development platforms and documentation is scattered. So it’s hard to find a good starting point.

We will start with the Gateway SDK. I want to make the usage of this SDK as easy as possible.

This blog is the first part of a series:

• Part one, how to use modules, gateway configuration and the broker
• Part two, connecting to the IoT Hub
• Part three, receiving commands from the cloud
• Part four, IoT Hub Routing
• Part five, Watchdog
• Part six, Modbus for Advantech Wise modules

Where to start?

The team behind the Gateway SDK is promoting the SDK at multiple locations on the internet:

• The official advertisement 🙂
• The Open source repository
• The documentation
• There are even examples available, both with simulated and a real device

That’s great but I am also forced to think about which language to use:

• Java
• NodeJS
• .Net (native framework)
• .Net Core/Standard
• C

This was very confusing at first.

Now knowing how the gateway is organized and having it running on an actual device, connected to real sensors, it’s less overwhelming.

In this series, we will use the .Net native framework version. I have an actual gateway running Windows 7 Embedded so this was my first choice.

The architecture, sort of

Before we look at the actual coding, first we check out the architecture. This is a simplified model of the gateway:

The three blocks are called ‘modules’. There are various pre-compiled/pre-programmed modules already available for various operating systems. and you can build them yourself. In our case, in .Net, these modules are available as classes in DLL’s.

But DLL cannot run automatically. To run these modules, we need a runtime:

This is a more detailed architecture. A broker exchanges messages from module to module. Messages are filled with content which is actual data from (simulated) devices. It must be clear modules are put after each other with a certain intent. Modules cannot be hustled. Each module has a certain job (eg. ingests, filters, logs, sends data to an IotHub, has knowledge about device connection strings) so there are rules on how to use them.

It must be clear modules are put in a certain order (and passing messages) with a certain intent. Modules cannot be hustled. Each module has a certain job (eg. ingests, filters, logs, sends data to an IotHub, having knowledge about device connection strings) so there are rules on how to use them.

In our case, we need a runtime named ‘gw.exe’, a console app. Later on, we will see how we get that one. And when this runtime is started, it needs some configuration, written in a file.

In this configuration file, having a specified format, two sections are available:

1. In the first section all modules, which need to be loaded, are described including the configuration for each module. For example, a logging module needs to know the location of the file to write to
2. In the second section, the linking order of modules is shown. So the runtime knows how to pass messages from one module to one or more other modules.

This is a simple broker concept, now let’s look at the messages passed:

public class Message
{
  ...
  public byte[] Content { get; }
  public Dictionary<string, string> Properties { get; }
  ...
}

A message is sent between modules. It must be clear a regular message contains two parts:

1. The content. This is the actual message eg. coming from a device containing telemetry (temperature, humidity, etc.)
2. The properties, a dictionary of key/value pair. This is meta data and it is used by one module to give guidance to the next module

Normally, one module fills the content and maybe it adds some guidance to the next module. This second module checks the properties and passes the content to the next module.

Note: so modules read the properties and maybe add some and/or deletes some. For an extended example, look at the documentation of the mapper module. We will look at this module in detail on the next blog.

Let’s start building a gateway

This is a nice story. But where to start? There are already some (pre-configured) examples provided by Microsoft, here we build one from the ground up.

Open Visual Studio 2017 and start a new solution containing a … .Net Framework class library:

Then, add a NuGet gateway package using the filter ‘azure.devices.gateway‘:

Note: pick the one for native windows

Once the NuGet package is loaded, it seems no references are added! What was the content of this package, actually?

Just compile the (empty) library and navigate to the \bin\debug output path:

Wut?

What we see here is a combination of the runtime (gw.exe) and a complete collection of pre-compiled and ready-to-use modules. Most of these modules will be handled on my next blog.

Ok, we have a class library in our solution and we have a gw.exe in the bin\debug folder. The class library does not run directly so what now?

Transforming our class library into a gateway

The answer is simple, we alter the start action of our class library. Go to the properties of the class library and open the Debug tab:

Alter two things:

1. Select ‘start external program’ and navigate to the ‘gw.exe’ in the bin\debug folder
2. Add a command line argument ‘modules.json’

From now on, if we compile and start our class library, we will actually start the executable.

In the previous paragraph, we learned the gateway needs both (compiled) modules and a configuration file. The modules are provided in the same folder, reflection can be used to get them. But the gw.exe needs a JSON file:

Usage:
gw <path to config json>

So the next step is to construct this gateway configuration file.

Gateway configuration

The configuration file is just a JSON file. And to demonstrate how to fill it, I first looked at the list of available modules:

• identity_map.dll
• iothub.dll
• logger.dll
• simulated_device.dll

Maybe there are even more modules but these I recognize from the SDK module list.

Just to show how modules behave, I only picked the ‘simulated device’ module which generated some telemetry. And we will write a new module using C# which writes the message received from the simulated device into the console.

So the config file needs the following parts:

• A description of the ‘simulated device’ module and the arguments it needs
• A description of our (soon to build) print module
• A link between these two modules so the gateway knows how to pass the messages

This is the complete configuration:

{
  "modules": [
    {
      "name": "simulator",
      "loader": {
        "name": "native",
        "entrypoint": {
          "module.path": "simulated_device.dll"
        }
      },
      "args": {
        "macAddress": "01:01:01:01:01:01",
        "messagePeriod": 5000
      }
    },
    {
      "name": "dotnet_printer_module",
      "loader": {
        "name": "dotnet",
        "entrypoint": {
          "assembly.name": "MyFirstEdge",
          "entry.type": "MyFirstEdge.MessagePrinterModule"
        }
      },
      "args": ""
    }
  ],
  "links": [
    {
      "source": "simulator",
      "sink": "dotnet_printer_module"
    }
  ]
}

Note: although defined, our own module is not built yet so do not expect this will execute already. Be patient 🙂

We can make several observations:

• modules do have a simplified name which is used in the links
• the args is a just JSON notation with several arguments for that module
• the arguments of the ‘simulator’ are taken from the documentation
• The simulated device provides a new message every five seconds
• the simulated device has a MAC address with a unique identifier. In a next blog, we will see why Microsoft has taken this step
• the entry point is very specific about which class we use. I named my library ‘MyFirstEdge’, the namespace also ‘MyFirstEdge’ and the class (still to write) ‘MessagePrinterModule’
• the simulated device is put in a ‘native’ library, our class is put in a ‘dotnet’ library

So save this JSON file named ‘modules.json’ in the project and mark it as ‘copy always’:

Next time we compile our library, the JSON file is put next to ‘gw.exe’ and our gateway is ready to run.

We need one more thing to do, writing our own module.

Just another printer module

Yes, this printer module is not unique.There are several others to be found in the documentation. But my module is a little bit more descriptive and therefore more useful compared to other blogs!

Add a class file and fill it with this code:

using Microsoft.Azure.IoT.Gateway;
using System;
using System.Text;

namespace MyFirstEdge
{
  public class MessagePrinterModule : IGatewayModule, IGatewayModuleStart
  {
    private string configuration;

    public void Create(Broker broker, byte[] configuration)
    {
      this.configuration = Encoding.UTF8.GetString(configuration);
    }

    public void Start()
    {
      Console.WriteLine("MessagePrinterModule started");
    }

    public void Destroy()
    {
    }

    public void Receive(Message received_message)
    {
      var message = $"Printer message: {Encoding.UTF8.GetString(received_message.Content, 0, received_message.Content.Length)}";

      var properties = "Printer properties: ";

      foreach (var property in received_message.Properties)
      {
        properties += $"Key '{property.Key}' / Value '{property.Value}'; ";
      }

      Console.WriteLine($"{message}\n{properties}");
    }
  }
}

Again, there are several observations to be made:

• please check the namespace. It must be on par with the entry point in the config file
• The module implements IGatewayModule so it has several methods: Create, Destroy and Receive. and it implements IGatewayModuleStart so it contains the Start method
• The methods are on par with the life and death of the module during creation, starting, receiving messages and being destroyed. Once started, the modules are not stopped anymore
• The configuration can be read as a string in the Create method. It should be easy to read these arguments as a JSON string and decode it into a class
• The message is a combination of both the message content and the properties (the context). So two console lines are written on the screen for each message received
• We need to add another NuGet package: ‘Microsoft.Azure.IoT.Gateway.Module‘. This one provides the interfaces we implement in the module

We are now ready to run our gateway.

Note: during execution, we can still debug the module. Add a breakpoint if you want to.

Bringing all pieces together

Run the gateway:

The gateway loads the modules and starts the printer module. Every five seconds a new message is provided by the simulated device module and picked up by the printer module.

We can see the message content: “{“temperature”: 10.00}” which is correct JSON message.

And the properties are:

1.  ‘source’ = ‘bleTelemetry’
2. ‘macAddress’ = ’01:01:01:01:01:01′

In the next blog, we will see that multiple modules provide a property ‘source’. This way, a module which receives a message can trace back where the message is coming from.

Note: this printer module could be placed after multiple modules

You can stop the gateway just by giving the DOSBox an ‘enter’.

Conclusion

Let’s recap:

• We created a class library
• We added a NuGet package Azure.Devices.Gateway containing the gw.exe runtime
• We altered the properties to start with that external program
• We added a configuration file which describes both the modules used and the link between them
• We added a class which is a printer module
• We added a NuGet package Microsoft.Azure.IoT.Gateway.Module so we could implement gateway module interfaces

That was not so hard, after all. And now we all know which parts a gateway is made of.

In my next blog, we will actually send some telemetry to an IoT Hub!

 

Advertentie