Take back control over IoTHub messages in Azure Functions

Azure Functions are a blessing for IoT solutions. To be so flexible executing code whenever messages are arriving, every IoT project is fully depending on it.

But one of the biggest frustrations is the casting of (EventHub) messages towards a string! Only the message body is left! Once a message is passed on to an Azure Function, I only have access to the body of the message. I can not access the (routing) properties anymore.

And before we got Azure Functions, we had to work with Stream Analytics. And I still do! And it’s so nice to have access to the IoT Hub values like the device name of the message. Because I am working with Azure Functions, I have to put it in the Message body first???

It would be great to have access to both the properties and the IoTHub values!

Well, it’s possible now with some clever casting…

Current Azure Function for IoTHub  triggers a string

Look at this message, It’s coming from a Modbus module from IoT Edge:

2/13/2018 6:34:17 PM> Device: [UNO2271GSV], Data:[[{"DisplayName":"RelayOne","HwId":"Wise4012E-142","Address":"00017","Value":"1","SourceTimestamp":"2018-02-13 17:34:12"},{"DisplayName":"KnobTwo","HwId":"Wise4012E-142","Address":"40002","Value":"4573","SourceTimestamp":"2018-02-13 17:34:14"},{"DisplayName":"KnobOne","HwId":"Wise4012E-142","Address":"40001","Value":"4587","SourceTimestamp":"2018-02-13 17:34:16"},{"DisplayName":"SwitchOne","HwId":"Wise4012E-142","Address":"00001","Value":"1","SourceTimestamp":"2018-02-13 17:34:16"},{"DisplayName":"RelayTwo","HwId":"Wise4012E-142","Address":"00018","Value":"1","SourceTimestamp":"2018-02-13 17:34:17"}]]Properties:
'content-type': 'application/edge-modbus-json'

The message is passed on by device “UNO2271GSV” and it has a property named “content-type” (and with the value is “application/edge-modbus-json”).

I pass it on to the following Azure Function. The parameters are just generated for me by the Azure Function template wizard for the IoT Hub trigger:

#r "Newtonsoft.Json"

using Newtonsoft.Json;
using System;

public static void Run(string myIoTHubMessage, TraceWriter log)
  log.Info($"C# IoT Hub trigger function processed a message: {myIoTHubMessage}");
  var list = JsonConvert.DeserializeObject<dynamic>(myIoTHubMessage);
  foreach (var item in list)
    log.Info($"{item.HwId} {item.DisplayName} {item.Address} {item.Value}");

Once the messages arrive, I can extract the values with some cleaver JSON deserialization from the body only:

2018-02-13T18:09:23.451 Function started (Id=93ffb6cb-7923-4e3d-b0b5-b59e3c94d535)
2018-02-13T18:09:23.451 C# IoT Hub trigger function processed a message: [{"DisplayName":"SwitchTwo","HwId":"Wise4012E-142","Address":"00002","Value":"1","SourceTimestamp":"2018-02-13 18:09:17"},{"DisplayName":"RelayOne","HwId":"Wise4012E-142","Address":"00017","Value":"1","SourceTimestamp":"2018-02-13 18:09:18"},{"DisplayName":"KnobTwo","HwId":"Wise4012E-142","Address":"40002","Value":"4573","SourceTimestamp":"2018-02-13 18:09:19"},{"DisplayName":"SwitchOne","HwId":"Wise4012E-142","Address":"00001","Value":"1","SourceTimestamp":"2018-02-13 18:09:19"},{"DisplayName":"KnobOne","HwId":"Wise4012E-142","Address":"40001","Value":"4590","SourceTimestamp":"2018-02-13 18:09:21"},{"DisplayName":"RelayTwo","HwId":"Wise4012E-142","Address":"00018","Value":"1","SourceTimestamp":"2018-02-13 18:09:22"}]
2018-02-13T18:09:23.451 Wise4012E-142 SwitchTwo 00002 1
2018-02-13T18:09:23.451 Wise4012E-142 RelayOne 00017 1
2018-02-13T18:09:23.451 Wise4012E-142 KnobTwo 40002 4573
2018-02-13T18:09:23.451 Wise4012E-142 SwitchOne 00001 1
2018-02-13T18:09:23.451 Wise4012E-142 KnobOne 40001 4590
2018-02-13T18:09:23.451 Wise4012E-142 RelayTwo 00018 1
2018-02-13T18:09:23.451 Function completed (Success, Id=93ffb6cb-7923-4e3d-b0b5-b59e3c94d535, Duration=0ms)

But do you notice that I am unable to see from which device the message is coming?

And worse, I receive multiple types of messages from multiple sources which are all routed to the same function. The following message has a different message property and different body properties:

2/13/2018 7:13:19 PM> Device: [UNO2271GSV], Data:[[{"hwid":"Wise4012E-142","relay":"RelayTwo","action":"1"}]]Properties:
'source': 'ASA'
'OutputName': 'alertonesink'

But I’m not able to read the properties of that message. So the original code will throw an exception caused by the non-existing body properties:

2018-02-13T18:10:54.207 Function started (Id=82afa3f3-a887-419e-a6be-96f0acc2fba0)
2018-02-13T18:10:54.207 C# IoT Hub trigger function processed a message: [{"hwid":"Wise4012E-142","relay":"RelayTwo","action":"1"}]
2018-02-13T18:10:54.408 Exception while executing function: Functions.IoTHubCSharp_EventHubTrigger. mscorlib: Exception has been thrown by the target of an invocation. Anonymously Hosted DynamicMethods Assembly: 'Newtonsoft.Json.Linq.JProperty' does not contain a definition for 'HwId'.
2018-02-13T18:10:54.423 Function completed (Failure, Id=82afa3f3-a887-419e-a6be-96f0acc2fba0, Duration=226ms)

Wouldn’t it be nice to have access to these kinds of things?

Clever casting to the rescue

I was made aware by a friend we can cast the incoming message to an EventData class. If we look at this class, it could contain everything we need. so let us cast it:

#r "Newtonsoft.Json"
#r "Microsoft.ServiceBus"

using Microsoft.ServiceBus.Messaging; 
using Newtonsoft.Json;
using System;

public static void Run(EventData myIoTHubMessage, TraceWriter log)
  log.Info($"C# IoT Hub trigger function processed a message: {myIoTHubMessage}");
  var bodyText = string.Empty;
  using(var stream = myIoTHubMessage.GetBodyStream())
  using(var streamReader = new StreamReader(stream))
    bodyText = streamReader.ReadToEnd();
    log.Info($"body {bodyText}");

  var systemProperties = string.Join(" - ", myIoTHubMessage.SystemProperties.Select(x => $"{x.Key} = {x.Value}"));
  log.Info($"System properties: {systemProperties}");

  var properties = string.Join(" - ", myIoTHubMessage.Properties.Select(x => $"{x.Key} = {x.Value}"));
  log.Info($"Properties: {properties}");

  if (myIoTHubMessage.Properties.Any(x => x.Key == "content-type"
                                          && x.Value.ToString() == "application/edge-modbus-json" ))
    var list = JsonConvert.DeserializeObject<dynamic>(bodyText);

    foreach (var item in list)
      log.Info($"{item.HwId} {item.DisplayName} {item.Address} {item.Value}");

We just cast the incoming message parameter. It’s no string anymore so we need a stream reader to get to the original body inside the new EventData.

But we also get access to the properties (see how I can check out the “content-type” in Properties).

And we also get access to the original IoTHub properties. Check out the “iothub-connection-device-id” in  SystemProperties:

id="log-stream" class="log-stream bottom-logs">2018-02-13T18:38:26.653 Function started (Id=0c56396d-94c9-4c22-bf98-ed184b75c4ab)
2018-02-13T18:38:26.653 C# IoT Hub trigger function processed a message: Microsoft.ServiceBus.Messaging.EventData
2018-02-13T18:38:26.653 body [{"DisplayName":"KnobTwo","HwId":"Wise4012E-142","Address":"40002","Value":"4581","SourceTimestamp":"2018-02-13 18:38:21"},{"DisplayName":"SwitchOne","HwId":"Wise4012E-142","Address":"00001","Value":"1","SourceTimestamp":"2018-02-13 18:38:22"},{"DisplayName":"SwitchTwo","HwId":"Wise4012E-142","Address":"00002","Value":"1","SourceTimestamp":"2018-02-13 18:38:23"},{"DisplayName":"KnobOne","HwId":"Wise4012E-142","Address":"40001","Value":"4581","SourceTimestamp":"2018-02-13 18:38:24"},{"DisplayName":"RelayTwo","HwId":"Wise4012E-142","Address":"00018","Value":"1","SourceTimestamp":"2018-02-13 18:38:26"}]
2018-02-13T18:38:26.653 System properties: iothub-connection-device-id = UNO2271GSV - iothub-connection-module-id = msftmodbus - iothub-connection-auth-method = {"scope":"module","type":"sas","issuer":"iothub","acceptingIpFilterRule":null} - iothub-connection-auth-generation-id = 636538553662468964 - iothub-enqueuedtime = 2/13/2018 6:38:26 PM - iothub-message-source = Telemetry - x-opt-sequence-number = 138869 - x-opt-offset = 8621147216 - x-opt-enqueued-time = 2/13/2018 6:38:26 PM - EnqueuedTimeUtc = 2/13/2018 6:38:26 PM - SequenceNumber = 138869 - Offset = 8621147216
2018-02-13T18:38:26.653 Properties: content-type = application/edge-modbus-json
2018-02-13T18:38:26.653 Wise4012E-142 KnobTwo 40002 4581
2018-02-13T18:38:26.653 Wise4012E-142 SwitchOne 00001 1
2018-02-13T18:38:26.653 Wise4012E-142 SwitchTwo 00002 1
2018-02-13T18:38:26.653 Wise4012E-142 KnobOne 40001 4581
2018-02-13T18:38:26.653 Wise4012E-142 RelayTwo 00018 1
2018-02-13T18:38:26.653 Function completed (Success, Id=0c56396d-94c9-4c22-bf98-ed184b75c4ab, Duration=0ms)

We can code against both the system properties (like the IoTHub device name) and message properties (like the context). Here we ignore any message with other body properties:

2018-02-13T18:49:02.819 Function started (Id=c97e43cb-853f-4a40-87ef-27d987ebb4f7)
2018-02-13T18:49:02.819 C# IoT Hub trigger function processed a message: Microsoft.ServiceBus.Messaging.EventData
2018-02-13T18:49:02.819 body {"HwId":"Wise4012E-142","UId":"1","Address":"00017","Value":"1"}
2018-02-13T18:49:02.819 System properties: iothub-connection-device-id = UNO2271GSV - iothub-connection-module-id = alert - iothub-connection-auth-method = {"scope":"module","type":"sas","issuer":"iothub","acceptingIpFilterRule":null} - iothub-connection-auth-generation-id = 636532063106016321 - iothub-enqueuedtime = 2/13/2018 6:49:02 PM - iothub-message-source = Telemetry - x-opt-sequence-number = 139400 - x-opt-offset = 8621475912 - x-opt-enqueued-time = 2/13/2018 6:49:02 PM - EnqueuedTimeUtc = 2/13/2018 6:49:02 PM - SequenceNumber = 139400 - Offset = 8621475912
2018-02-13T18:49:02.819 Properties: command-type = ModbusWrite
2018-02-13T18:49:02.819 Function completed (Success, Id=dbea372f-8147-4d06-ba7e-94c391b965e6, Duration=0ms)


Finally, we are no longer dependent on the IoTHub routing is we want to know which message properties are accompanying the message. And we have all the tools in our hands to access the device name (and other goodies) in the system properties.

Bonus: Execution context

Did you ever wonder about the execution context of Azure functions? Maybe you are also confused like me when the logging text from multiple messages cross each other?

Now you can fix this. Just add this extra “ExecutionContext” parameter to the Azure Function:

public static void Run(EventData myIoTHubMessage, TraceWriter log, ExecutionContext ec)
  var invocationId = ec.InvocationId.ToString();
  log.Info($"Executed in contect - {invocationId}");

Now you can correlate your own texts with the correct execution of an Azure Function.