How to implement module sideloading in Azure IoT Edge

As seen in the last couple of blogs, Azure IoT Edge is useful as a deployment vehicle for Docker containers.

The idea is that the default available edgeAgent module makes an outbound connection with the cloud so it can retrieve updated deployment manifests.

Once such a manifest is received, it will pull new or updated containers from their container repository (eg. hub.docker.com or your own Azure Container Registry).

Relying on Docker modules is smart: it’s flexible and reliable.

But there is one drawback: the size of containers. Containers on their own can have substantial sizes, one hundred megabytes in size is not uncommon.

Docker is a little bit smarter to cope with these sizes. Docker makes use of layers. Layers already downloaded are not downloaded again.

But still, if you are limited in the daily communication size (like on a satellite channel) or if you have to pay for the transfer of files over a metered network (like 3G/4G), it would be nice if you could ‘side-load’ these images to your Edge device.

The easiest way to side-load is during the assembly of your edge device while it’s made ready for shipping to the production site. Next to installing the Azure IoT Edge runtime and the moby container logic, you can already try to pull your containers as much as possible.

Once shipped, your containers are already in the local repository.

After that, you are NOT on your own.

Azure IoT Edge has a nice setting for controlling the pull mechanism.

Let’s check out how it works.

The idea is that we stop the edgeAgent from pulling new modules described in deployment manifests.

This means that the edgeAgent knows which modules have te be running but will not actively pull them from any container registry except for the local container registry!

It’s up to you to get these images loaded into the local repository. Once the image arrives there, the edgeAgent picks it up and load it in your Azure IoT Edge.

Sideloading

You have two options to sideload:

  1. you can perform your own ‘sudo docker pull [container tag]’ once you have access to unlimited communication (like a temporary WIFI connection)
  2. You can make use of a ‘sneaker network’ where containers are loaded by hand from a medium like a memory stick. Docker provides a save and load mechanism for that

We use this in combination with the edgeAgent property:

"imagePullPolicy": "never"

The default setting is ‘on-create’. But we want to suppress this pull behavior.

Let’s see this in action with a couple of steps.

Note: Although this property is present in the documentation for quite a while, I just got it demonstrated in IoT Edge version 1.0.9.

Note: If you make use of a ‘docker pull’ just fast forward to step two.

Step one: saving a module as tar file and stopping the runtime

For this demonstration, I have set up an Azure VM with Ubuntu and the IoT Edge runtime first. I updated the IoT Edge runtime and edgeAgent and edgeHub modules to version 1.0.9.

I demonstrate side-loading the well-known ‘SimulatedTemperatureSensor’ module. So To get my hands on the tar file of the container, I first pulled it using that IoT Edge:

As we can see, the simulation is running.

We can save the Docker container to a tar file on the filesystem:

sudo docker save mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0 -o sts.tar

Here we execute the command:

This result in a sts.tar on disk:

Once we have this file, we remove all traces of this simulation from the filesystem.

To prevent the IoT Edge Runtime to reload the module again, we stop the runtime temporarily:

sudo systemctl stop iotedge

This is the first step in removing the container from the system:

We can check the local repository with:

sudo docker images

This shows the currently loaded Docker images on the filesystem and their unique hashes.

We remove the simulation image with:

sudo docker rmi [hash of the simulation module]

After that, the module is gone:

We are now almost ready to start sideloading.

Step two: Updating the deployment manifest

If we restart the IoTEdge runtime at this point, the edgeAgent would load the temperature simulation as soon as possible.

We have to disable the pull policy:

"imagePullPolicy": "never"

This is a property of both the edgeAgent and separate modules.

I enabled it on both the edgeAgent and the sensor simulation.

Note: altering properties in the deployment manifest is NOT possible in the Azure Portal. You have to create a deployment manifest using Visual Studio Code. Once the updated manifest is deployed to the device, the updates in the deployment manifest will be honored by the Azure portal.

This is how the update looks like for the edgeAgent:

The full deployment manifest looks like:

{
    "modulesContent": {
        "$edgeAgent": {
            "properties.desired": {
                "modules": {
                    "SimulatedTemperatureSensor": {
                        "settings": {
                            "image": "mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0",
                            "createOptions": ""
                        },
                        "type": "docker",
                        "version": "1.0",
                        "status": "running",
                        "restartPolicy": "always",
                        "imagePullPolicy": "never"
                    }
                },
                "runtime": {
                    "settings": {
                        "minDockerVersion": "v1.25"
                    },
                    "type": "docker"
                },
                "schemaVersion": "1.0",
                "systemModules": {
                    "edgeAgent": {
                        "settings": {
                            "image": "mcr.microsoft.com/azureiotedge-agent:1.0.9",
                            "createOptions": ""
                        },
                        "type": "docker",
                        "imagePullPolicy": "never"
                    },
                    "edgeHub": {
                        "settings": {
                            "image": "mcr.microsoft.com/azureiotedge-hub:1.0.9",
                            "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}]}}}"
                        },
                        "type": "docker",
                        "status": "running",
                        "restartPolicy": "always"
                    }
                }
            }
        },
        "$edgeHub": {
            "properties.desired": {
                "routes": {
                    "route": "FROM /messages/* INTO $upstream"
                },
                "schemaVersion": "1.0",
                "storeAndForwardConfiguration": {
                    "timeToLiveSecs": 7200
                }
            }
        },
        "SimulatedTemperatureSensor": {
            "properties.desired": {
                "SendData": true,
                "SendInterval": 5
            }
        }
    }
}

Save the deployment manifest and deploy it to our IoT edge device (Note: the runtime is disabled at this point)

Step three: Starting the runtime

It’s time to start the runtime again:

sudo systemctl start iotedge

Once started, check the log of our edgeAgent:

sudo iotedge logs -f edgeAgent

This should look like:

As you can see, the edgeAgent is complaining about the missing module.

But: it is not starting a download!

This is now up to you.

If you want to perform the pull yourself, execute:

mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0

Or you can go for loading the tar file on the file system:

sudo docker load --input sts.tar

The second action will result in this log where the module is loaded into te repository:

And both actions will show the image in ‘sudo docker images’ afterwards.

If we check the edgeAgent log again with ‘sudo iotedge logs -f edgeAgent’ we will notice the container is picked up:

And actually, the module is running fine again:

We see the simulation is up and running.

Conclusion

We have proven we can sideload containers ourselves without affecting the execution of logic on the IoT Edge.

We are now able to save and load containers and we can use a ‘second channel’ to distribute the modules.

Further optimization could be possible if the Docker layers are taken into account.

It’s now up to you to orchestrate this sideload distribution.

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen.

WordPress.com logo

Je reageert onder je WordPress.com account. Log uit /  Bijwerken )

Google photo

Je reageert onder je Google account. Log uit /  Bijwerken )

Twitter-afbeelding

Je reageert onder je Twitter account. Log uit /  Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit /  Bijwerken )

Verbinden met %s