Deploy an Azure StreamAnalytics Job on the IoTEdge

[Update 16-01-2018]: There is a workaround available. Log into the Azure Portal using:  https://portal.azure.com/?Microsoft_Azure_StreamAnalytics_onedge=true

[Update 15-01-2018]: The issue is being fixed asap. I have been notified the dialogs of Azure Stream Analytics will be changed in a couple of days. This blog will remain valid, though.

[Update 13-01-2018]: It seems the IoTEdge-ASA integration is suddenly not available in the Azure Portal. Eg. the Environment Host question shown below is dropped and no Edge Hub input or output is selectable anymore. No idea why… When more information is available, a new update will be posted here.

The second version of the Microsoft IoTEdge solution is now available as Public Preview. In this version, you can run predefined modules like Modbus,  build your own modules, deploy Azure Machine Learning modules, deploy Azure Functions and you can deploy Azure StreamAnalytics jobs.

The concept is pretty simple, as always you have to create an ASA job, define inputs, outputs and a query. But this time the ASA will run on a local device:

Microsoft provides documentation here and here explaining how to deploy your ASA modules. Let’s dig a bit deeper into it.

Advertentie

Introduction to the microsoft/azureiotedge-modbus-tcp IoTEdge Module

The newest version of the Azure IoTEdge solution is a very promising platform. The combination of remote provisioning the modules, the power of twin configuration and the new routing is interesting. But the learning curve is pretty steep.

The first version was based on programming an application. The new version is based on docker images, each being a separate application, which has to be stored in the container registry of your choice (like Docker Hub or your own container registry in Azure.

So once you have learned how to build and deploy your own modules, you can check out the modules Microsoft already supplies.

One of these modules is a Modbus module. It’s available at the Docker Hub of Microsoft. Modbus is a great protocol for highspeed communication over TCP and I have already blogged about it, using the previous IoTEdge SDK version.

Let’s check out how we get some telemetry from it.

Better property roundtrip for Azure IoTEdge Module Twins

The newest version of the Azure IoT Edge is still in public preview but this one makes the Edge truly intelligent!

We are now able to both distribute and manage logic on-premise.

Version one of the IoT Edge was and still is based on running an executable. And each module (ingest, transform, distribute) is a class in a dynamic library. And all modules are connected using a broker serving the role as MessageBus. Messages between modules are exchanged using configurable routes.

This first version is still available for download. But the new version is fundamentally different.

Modules hosted as a container

In the new version, we still see modules connected by and exchanging messages using a local MessageBus. But Microsoft has redefined the modules into separate executables, each running is a different container. The containers are hosted in Docker and can run on Linux or Windows.

Microsoft already provided a lot of modules in the Docker Hub, for OPC UA support, for Modbus support, etc. And you can deploy your own modules too!

There are multiple examples available on the Microsoft website which explain how to get started.

In this blog, I show how tweaking some example code leads to even better management and a better understanding of what’s happening on the Edge using the Module Twins properties.

Forget about messages, just care about IoTHub chunks

Ok, this blog is all about taking the blue pill or the red pill:

I use the Azure IoTHub for every project I am involved in. It’s my main focus on getting IoT things done and using this tool is how I manage.

And consequently, I always look at the number of daily messages we have to ingest because the price of an Azure IoTHub depends on the number of messages it passes on.

It starts with 400.000 messages a day and from there, you can scale up. The biggest version can scale up to a tenfold of 300 million messages a day:

But this is not the whole truth, you can send more…

Zip and unzip a string of data in memory

I was looking for a way to zip and unzip a file without a filename…

Normally, this is not really an issue. There are lots of libraries and NuGet packages available.

But this file was represented to me by code as a string, some JSON, actually.

So I was looking for a way to convert a string into some zipped format and back. This was harder than expected.

In the end, I wrote a solution myself. Here it is for you to use it, just to save you a lot of time.

Note: This library is available on GitHub. And Visual Studio users can import the NuGet package.

UPDATE: Recent additions include support for GZIP and Deflate compression and decompression. This is based on byte arrays instead of strings to make it more versatile. Also, the string encoding is preserved inside a byte array.

Migrating Azure Functions from portal to Visual Studio

Azure Functions are a great addition to the whole family of Azure resources. Azure Functions are my number one workhorses. The fact that I can execute code, triggered by ‘whatever’, gives me so much power for a fraction of the costs.

And the beauty of it is, you can write those nifty pieces of code directly into the Azure portal using nothing more than a browser.

But as always, with great power comes great responsibilities!

The code I write in the portal is not supporting any version control. And I cannot debug this code.

Although I am up-and-running with Azure Functions in a few hours, I want to debug and put my code in version control too!

Luckily, I can create Azure Function in Visual Studio too. And this gives me the power of debugging and version control. But what are the drawbacks?

Let’s explore om how to migrate your Azure Function, written in the portal, into Visual Studio.

Tweede Azure IoT Community bijeenkomst op 1 februari 2018

Op 1 februari 2018 wordt al weer de tweede Azure IoT Community bijeenkomst georganieerd!

Na de succesvolle eerste bijeenkomst wordt het weer tijd voor een nieuwe mogelijkheid om elkaar te ontmoeten.

Deze avond is voor iedereen die geinteresseerd is de combinatie van Microsoft, Windows, Azure en natuurlijk IoT. Het maakt niet uit of je nu hobbyist, student of professional bent. Kom lang, neem je vrienden mee en deel jouw ervaringen of doe juist ervaring op.

Het programma voor deze avond is:

17.30 – Inloop en eten

18.30 – Korte introductie en laatste updates van Microsoft IoT producten

18.40 – Industrial IoT with Azure IoT Hub – door Eldert Grootenboer / Motion10

19.30 – Alles over de Azure Stream Analytics

20.15 – Borrel

De avond is dit maal verzorgd door Atos, de locatie is Burgemeester Rijnderslaan 30 in Amstelveen (routebeschrijving)

Dus meld je aan op via deze link (en neem legitimatie mee).

Meer informatie is te vinden op onze website. Tot ziens op de volgende bijeenkomst!

 

Introduction to the IoT Edge SDK V1, part 6: Modbus for Wise modules

Update: Recently, Microsoft introduced the new V2 version of their IoT Edge. This blog is referencing the former but still supported V1.

I do a lot of IoT research using the Advantech Wise 4012E IO module. It’s biggest advantages are that it runs on 5 Volts, using a MicroUSB connection (instead of a bulky 12V or 24V adapter) and it comes with knobs, switches and LEDs to simulate real sensors. So it’s a very compact but yet complete IoT device.

Until a few days ago I made use of the Rest protocol to contact that module but this has some disadvantages.

First, Rest is a great protocol in the IT world but it is not used that much in the OT world. Luckily, the IO module also supports the Modbus protocol. So I tried to switch to that protocol.

Second, Rest is very slow compared to other protocols like Modbus. Using Rest, I’m lucky when I can pull 2 or 3 requests a second out of the module.

In the previous blogs, we have seen multiple modules, both provided by Microsoft or created on the fly. And Microsoft provides a genuine Modbus module for the IoT Edge SDK. There is only one drawback, it’s on Github but it’s not available as NuGet package. You have to make/build it yourself!

And for some unknown reason, I did not get it working the way I liked it. I encountered too many exceptions.

Update: There is also a Version 2 of this module. In this blog, I refer to V1.

So, in the end, I just ignored the module and build my own Modbus module.