Add cloud logging to Azure IoT Edge

Azure IoT Edge is an interesting platform for Edge computing. It opens up a lot of new scenarios for local computing as an extension of the Azure cloud. And the combination is both secure and flexible due to the usage of open (security) standards.

But it’s still in Public Preview so once in a while things go sour. Either no messages arrive at in the cloud or local logic is not executed. So this means you have to log into your Edge device (remember to always have a backup plan) and check out the local logging.

But what if I could check out the logging right within Azure?

I came across this Github gem to make Azure IoT Edge local logs available in Azure! And it only takes 15 minutes or less.

“This repository provides an Azure IoT Edge module that can be used to send container logs from other modules on the edge device, including the edge runtime, securely to Azure Log Analytics in the cloud. “

How to tag your IoT Hub Devices

Until now, device twin tags were a bit lame.

Yes, desired and reported properties were much more fun to play with.

But those of you who administer thousands of Azure IoT devices, you really appreciate tags. It’s the only way to control that large amount of devices without losing your head.

Why? Because you first query your devices and then execute jobs on these subsets.

And Microsoft is making use of this feature a lot. You will have to use tags if you want to execute IoT Edge deployments (still in preview) or if you want to use the recently added Automatic Device Management (even newer):

But how do you actually add or alter tags of devices? What tooling is Microsoft providing?

Let’s check out a number of ways to tag and start querying your devices.

Your Docker container takes care of persisting your SQL Server database

Microsoft has a great solution for persisting your local data collected by the IoT Edge. It’s up to you to get the data in and out of your database. But you have a convenient way of storing your precious information into some kind of persistent storage.

As seen in my previous blog, it’s not that hard to deploy from the cloud and administer your SQL database locally.

But how persistent is your database actually? Can you trust Docker for taking care of your data?

In this blog, we try to answer this question.

Administer your SQL Server in Docker

As with many things, you have to do it, to believe it.

The same goes for the Azure IoT Edge solution.

With the new IoT Edge solution, Microsoft provides a platform, both powerful and scalable. It’s based on Docker and logic is put in Docker containers.

And in many presentations, this (correct) picture is used to show the capabilities:

I am already working with the IoT Edge preview quite some time and there is a ‘weak’ spot in the image.

I already marked it in red so it’s not that hard to find.

On several occasions, non-technical people explained the local storage as being a database to persist data from internal logic.

I understand the confusion, but this is just a ‘database’ used by the IoT Edge internals (I assume mostly the Edge Hub module) and it is not accessible by users. I know for sure it’s used for the store-and-forward pattern used to send ‘upstream’ messages to the cloud.

Once a message is routed to be sent to the cloud, it’s first stored by the EdgeHub. If the connection to the Azure IoT Hub cannot be established, the message is stored with a certain retention time ( see its configuration “storeAndForwardConfiguration”: { “timeToLiveSecs”: 7200} ).

So, how can we add local storage to our IoT Edge if we want to do something with custom code and databases, etc.

Well, Microsoft already has written a great piece of documentation here.

There, you can see how you create an SQL Server database both on Linux and Windows containers and you learn how to create a database and a table inside it. Finally, you access it using some Azure function.

Let’s dig a bit deeper into this.

We will look on other  (simpler) ways to work with the database.