If you look at the routes page in Azure IoT Edge configuration wizard, what do you prefer?
The current notation:
Or do you prefer a flow chart like this:
The routes in Azure IoT edge are a clever solution to describe how messages from one module are sent to another. But the JSON notation can become less readable once you add more (up to twenty) modules. That could end up eg. nineteen routes or more!
Just as an experiment I was thinking about how the ease the experience using a graphical interface.
I prefer the second solution, probably just like you.
So let’s look at how you can create the same experience with your routes of your IoT Edge device.
The routes of an IoT Edge device behave like a flowchart, messages are just arrows between modules, linking them together. A colleague pointed me at http://www.webgraphviz.com/ but that did not give me the right route experience of inputs and outputs.
Then I came across this open source Jquery Flowchart, a javascript jQuery plugin that allows you to draw a flow chart. You can take a look at the demos.
This gave me almost everything I needed:
I can see:
- modules (operator)
- inputs
- outputs
- routes (link, an arrow)
It lacks a text in the arrow (eg. for the name of the route or the Where clause of the route). But it’s open source, you can add it yourself 🙂
How to visualize the routes
So we need to break down the coding of this visualization in small steps:
- We pick Asp.net MVC are the host of our project
- We have to get the jquery running in our Asp.net MVC project
- We must be able to download a route from a device using code-behind logic
- We have to break down the JSON routes in modules, inputs, outputs, and arrows
- We have to construct unique modules
- We have to fit these parts into something the flowcharts recognizes, JSON?
- We have to pass the JSON to the flowchart in the UI in the browser
- Prettify
So let’s start.
1 We pick Asp.net MVC are the host of our project
It’s not that hard to start with ASP.Net MVC. Just open Visual Studio, create an ASP.Net MVC website using the template (the one with the index page already generated) and run it once, just to make sure everything (code, compile, web server, browser) is working fine.
2 We have to get the JQuery running in our Asp.net MVC project
The jquery UI flowchart javascript and stylesheet are coming from GitHub. Just add the ‘jquery.flowchart.css’ (not the minified version, for now, we have to fix something) and the ‘jquery.flowchart.js’ in the Index.cshtml.
I added them to the ‘wwwroot\js’ folder:
Just for convenience, I removed the references to Jquery and JQueryUI to this section also:
@{
ViewData["Title"] = "Iot Edge routes flow";
}
<div>
<h1>IoT Edge routes flow diagram</h1>
</div>
<div></div>
@section Scripts{
...
}
Note: we use a scripts section, we will add some javascript in there later on.
Note 2: the div ‘routingchart’ will contain the flowchart later on. The size is just set to give us a canvas.
3 We must be able to download a route from a device using code-behind logic
We know we want to get our hand on the routes of a device. How can we read it from an IoT Edge device?
We can find the routes being part of the desired device properties of the IoT Edge EdgeHub module:
We can access this using the Microsoft.Azure.Devices NuGet package:
So download the NuGet package, get the connection string of your IoT Hub (please be careful with it), select the name of your IoT Edge device, and download the routes into a JSON format:
public IActionResult Index()
{
var connectionString = "HostName=[iothub].azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=[key]";
var registryManager = RegistryManager.CreateFromConnectionString(connectionString);
var twin = registryManager.GetTwinAsync("LinuxArk1123", "$edgeHub").Result;
var desired = twin.Properties.Desired;
var routes = desired["routes"];
...
}
This method needs the connection string. I recommend using an Azure Key Vault instead of hardcoding it like this.
Note: This method will be extended with a few more lines later on. The full method is shown below (step 7).
4 We have to break down the JSON routes in modules, inputs, outputs, and arrows
So we have a website set up, we can extract the routes of a module by code, now comes the fun part: RegEx.
For me, RegEx has always been some kind of torture. This gibberish is superior over almost all other solutions to parse texts but it looks like digital vomit from a drunk programmer after 72 hours of continuously coding without any sleep.
But the notation of routes can become very diverse due to the fact you can leave out input names, output names or even module names. And there is also something like ‘upstream’.
Parsing routes screams for RegEx with groups so I started up http://regexstorm.net/tester. And after several experiments, I came up with a plausible regular expression:
I had to come up with a second regex so I could support ‘upstream’ too!
And this is the code to parse routes:
private static List ExtractEdgeHubRoutes(dynamic routes)
{
var routeList = new List();
foreach (var r in routes)
{
var key = r.Key;
string value = r.Value;
var regex1 = new Regex(@"modules/(\w+)[/outputs]{0,}/(.+?|\*)\b.*INTO.*modules/(\w+)/inputs/(.+?|\*)""", RegexOptions.IgnoreCase);
var match1 = regex1.Match(value);
if (match1.Success
&& match1.Groups.Count > 4)
{
routeList.Add(
new Route
{
Id = r.Key,
ModuleFrom = match1.Groups[1].Value,
Output = match1.Groups[2].Value,
ModuleTo = match1.Groups[3].Value,
Input = match1.Groups[4].Value,
});
}
var regex2 = new Regex(@"modules/(\w+)[/outputs]{0,}/(.+?|\*)\b.*INTO.*(\$upstream)", RegexOptions.IgnoreCase);
var match2 = regex2.Match(value);
if (match2.Success
&& match2.Groups.Count > 3)
{
routeList.Add(
new Route
{
Id = r.Key,
ModuleFrom = match2.Groups[1].Value,
Output = match2.Groups[2].Value,
ModuleTo = match2.Groups[3].Value,
Input = "upstream",
});
}
}
return routeList;
}
Note: We have to escape som characters in the regex text.
Now we can parse the routes lines (taken from the dynamic) and we return a list of Route instances:
public class Route
{
public string Id { get; set; }
public string ModuleFrom { get; set; }
public string ModuleTo { get; set; }
public string Input { get; set; }
public string Output { get; set; }
}
The $Upstream module, which sends all messages it receives straight to the Azure IoTHub, has no actual input defined by default. So I add this imaginary input.
5 We have to construct unique modules
The routes are retrieved but the data is not structured, the same module names could be mentioned on multiple lines, just like the inputs and outputs. We need to know the unique modules and their specific inputs and outputs. Only then we can draw them in the flowchart:
private static List ExtractRoutes(List routeList)
{
var modules = new List();
var topIndex = 0;
var leftIndex = 0;
foreach (var route in routeList)
{
var moduleFrom = modules.FirstOrDefault(x => x.Title == route.ModuleFrom);
if (moduleFrom == null)
{
moduleFrom =
new Module
{
Id = Convert.ToString(route.ModuleFrom),
Title = Convert.ToString(route.ModuleFrom),
Top = topIndex,
Left = leftIndex,
};
modules.Add(moduleFrom);
topIndex = topIndex + 100;
leftIndex = leftIndex + 200;
}
var moduleTo = modules.FirstOrDefault(x => x.Title == route.ModuleTo);
if (moduleTo == null)
{
moduleTo =
new Module
{
Id = Convert.ToString(route.ModuleTo),
Title = Convert.ToString(route.ModuleTo),
Top = topIndex,
Left = leftIndex,
};
modules.Add(moduleTo);
topIndex = topIndex + 100;
leftIndex = leftIndex + 200;
}
if (!moduleFrom.Outputs.Any(x => x == Convert.ToString(route.Output)))
{
moduleFrom.Outputs.Add(Convert.ToString(route.Output));
}
if (!moduleTo.Inputs.Any(x => x == Convert.ToString(route.Input)))
{
moduleTo.Inputs.Add(Convert.ToString(route.Input));
}
}
return modules;
}
public class Module
{
public string Id { get; set; }
public string Title { get; set; }
public int Top { get; set; }
public int Left { get; set; }
public List Inputs { get; private set; } = new List();
public List Outputs { get; private set; } = new List();
}
Now we finally have a list of the individual modules, each with its unique input list and output list. And we already had the routes so we can start constructing the chart logic.
6 We have to fit these parts into something the flowcharts recognizes, JSON?
If we check out the examples from the creator of this flowchart library, the flowchart is able to ingest a JSON structure which describes the modules and the routes (the arrows).
So let’s construct that same structure with C# classes:
private static JsonObject ConstructFlowChart(List routeList, List moduleList)
{
var jsonObject = new JsonObject();
foreach (var route in routeList)
{
jsonObject.links.Add(
route.Id,
new Link
{
fromOperator = route.ModuleFrom,
fromConnector = route.Output,
toOperator = route.ModuleTo,
toConnector = route.Input,
});
}
foreach (var module in moduleList)
{
var o = new Operator();
jsonObject.operators.Add(module.Title, o);
o.top = module.Top;
o.left = module.Left;
o.properties.title = module.Title;
foreach (var output in module.Outputs)
{
o.properties.outputs.Add(output, new InputOutput { label = output });
}
foreach (var input in module.Inputs)
{
o.properties.inputs.Add(input, new InputOutput { label = input });
}
}
return jsonObject;
}
This returns a structure which is based on these classes:
public class JsonObject
{
public Dictionary operators { get; private set; } = new Dictionary();
public Dictionary links { get; private set; } = new Dictionary();
}
public class Operator
{
public Properties properties { get; private set; } = new Properties();
public int top { get; set; }
public int left { get; set; }
}
public class Properties
{
public string title { get; set; }
public Dictionary inputs { get; private set; } = new Dictionary();
public Dictionary outputs { get; private set; } = new Dictionary();
}
public class InputOutput
{
public string label { get; set; }
}
public class Link
{
public string fromOperator { get; set; }
public string toOperator { get; set; }
public string fromConnector { get; set; }
public string toConnector { get; set; }
}
This new class structure can be serialized into JSON in the same format as the Flow chart expects.
The idea to use dictionaries is taken from this JSON format manual.
7 We have to pass the JSON to the flowchart in the UI in the browser
Let’s return to the ‘Index’ method of the controller class which runs the code-behind of this web page:
public IActionResult Index()
{
var connectionString = "HostName=[iothub].azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=[key]";
var registryManager = RegistryManager.CreateFromConnectionString(connectionString);
var twin = registryManager.GetTwinAsync("LinuxArk1123", "$edgeHub").Result;
var desired = twin.Properties.Desired;
var routes = desired["routes"];
List routeList = ExtractEdgeHubRoutes(routes);
List moduleList = ExtractRoutes(routeList);
var jsonObject = ConstructFlowChart(routeList, moduleList);
var data = JsonConvert.SerializeObject(jsonObject);
return View("Index", data);
}
You can recognize how we read the desired properties to get to the route text lines.
And can recognize the various steps to turn the route text lines into a JSON structure the flow chart understands, a model.
This model is serialized into a string and Razor is used to make this structure part of the HTML we send to the browser.
In the browser, this model is then picked up by the javascript. This is the final version of the Scripts section:
@section Scripts{
$(document).ready(function () {
// The Model from the code-behind is used by the flowchart directly
var data = @Html.Raw(Model);
$('#routingchart').flowchart({
data: data,
multipleLinksOnInput: true,
multipleLinksOnOutput: true,
linkWidth: 3,
distanceFromArrow: 4
});
});
}
The model is added in the flow chart using the variable data.
But what is the usage of all the other properties the flowchart uses?
8 Prettify
By default, the flow chart only accepts one output link (arrow) to be connected to an input link. And an output could only have one connection to one input.:
- multipleLinksOnInput (default: false): Allows multiple links on the same input connector.
- multipleLinksOnOutput (default: false): Allows multiple links on the same output connector.
The changes to the lines (width) and arrows (distance) gives it a personal touch.
Finally, I asked you to get the non-minified version of the ‘~/js/jquery.flowchart.css’ file. Here is why.
The default width of the module boxes is too narrow to show the full name of the module or inputs or outputs. I changed the CSS so the boxes become wider:
Change it to the width you feel comfortable with.
The result
This is the end result after creating the flow chart and rearranging the modules by hand (just drag the modules to another location using the mouse):
The modules are wide enough for large module names and also large input names and output names fit next to each other.
Conclusion
This visualization makes it pretty clear how IoT Edge modules are joined together using the routes.
the origins flow chart library does not support a text inside a link (route) so we cannot at the optional Where clause of a route. This would be a nice addition.
This example shows only how to read and represent the routes configuration. It should be possible to also edit and extend the routes configuration. The original flow chart examples show how to add modules and how to add and delete routes. And it is possible to read the structure of the altered flow chart. So feel free to experiment and extend this experiment.
If you have created a visualization of your own module, please share it on twitter and include my twitter handle ‘@svelde’.
Update: You can try it yourself now at this site. The code is available as open source.
Sander van de Velde 