EventGrid namespace MQTT support, topics per client overview

The EventGrid namespace now offers a full vanilla MQTT Broker. This is a fine solution for connecting devices in a secure way to the cloud with custom MQTT topics. It can even provide Device-to-Device communication out of the box.

Just create the MQTT broker, enter the right information about clients, client groups, topic spaces, and permission bindings and you are good to go.

Are you?

Once you have set the clients, client groups, topic spaces, and permission bindings you probably have no clue anymore about what permissions each client has for what topics!

What if we could get a simple oversight like this:

Here you see all seven clients or an MQTT broker with their own topics and permissions.

I created a demonstrator for reading the Azure EventGrid namespace clients, client groups, topic spaces, and permission bindings. These are then combined to list all topics with their permissions for each MQTT broker client.

The full source is available on GitHub with an MIT license.

Let’s see how we can get an overview.

For a very elaborate overview of the MQTT broker capabilities, check out this blog post.

Goal

This blog post shows you how to use the sample GitHub repository to list all topics and permissions for each client in your EventGrid namespace-supported MQTT broker.

It relies on the ResourceManager EventGrid NuGet package and you can see how it’s done. Please start building your own logic using this knowledge.

Prerequisites

To follow along, you need to have an EventGrid Namespace:

• Create an EventGrid Namespace
• Activate MQTT on the Configuration tab page
• Create at least one client (optionally with a client attribute)
• Optionally create a client group
• Create a topic space with at least one topic template
• Create a permission binding

Note: For details on how to do this, check out this post.

From this EventGrid Namespace, collect these values:

• EventGrid Namespace name
• Subscription ID
• Resource group (name)

Notice these values can be found on the Overview tab page.

We are going to run a C# .Net console app. I demonstrate it in Visual Studio.

Finally, we need authorization permissions to read EventGrid Namespace information from Azure.

The demonstrator used CLI access so you need to install the Azure CLI (if not done already).

Once it is installed, run this and provide your credentials:

az upgrade
az login

Your Azure CLI is now logged in to Azure.

At this point, you have an EventGrid Namespace configured with topic bindings. You are also logged in to Azure via the Azure CLI.

Getting the repository

Go to the GitHub repository to get the source code:

Select the green button and make use of one of the options to download the content:

I clone it using the web URL:

https://github.com/sandervandevelde/MqttBrokerGraphApp.git

On your laptop, clone the repo by running the command on the dos prompt within a folder of your choice (eg. c:\git):

cd c:\git
git clone https://github.com/sandervandevelde/MqttBrokerGraphApp.git

If the Git CLI is not installed yet, get the installer here. You can also use Visual Studio Code if you prefer that excellent tool.

See the installation succeeds:

Now navigate to the ‘C:\git\MqttBrokerGraphApp\src’ folder with the source code and Visual Studio solution:

There are three projects:

• A library project
• A unit test project
• A demonstrator application

Start or open the Visual Studio solution ‘MqttBrokerGraphApp.sln’ file.

Running the C# .Net Demonstrator app

Once you have opened the Visual Studio solution, navigate to the ‘Program.cs’ in the ‘MqttBrokerGraphApp’ console app:

The application relies on three environment variables, equivalent to the three values taken from the Azure portal page:

• mqtt-graph-subscriptionid
• mqtt-graph-resourcegroupname
• mqtt-graph-namespacename

Add these three environment variables to your machine:

Notice that you probably need to restart Visual Studio so it reads the new environment variables!

Set the MqttBrokerGraphApp as a Startup Project and run it:

Here, I used another EventGrid namespace with four clients. The access to the topics is very clear.

Note: If a topic template uses a placeholder like ‘client.authenticationName’ as seen in ‘device/${client.authenticationName}/telemetry/#’ it is replaced with the actual device name.

Did you notice that the first EventGrid namespace (at the top of this post) had a client named ‘audit’ with publish and subscribe rights to ‘message/#’. This was unintentional. I should have only subscriber rights to ‘#’ (meaning all topics).

So, this tool makes it perfect to audit the rights of your clients.

If you want the authorization credentials provided by the application, use:

// Use host application access
var cred = new DefaultAzureCredential();

This single line makes use of the Azure portal/arm access via the credentials provided by the hosting application (eg. Visual Studio or an Azure Function).

How does it work?

The real magic is performed by the ‘Azure.ResourceManager.EventGrid’ NuGet package, used in the ‘DeviceClientQueryLibrary’ library.

Once the right Azure credentials are used, a way to read all Azure EventGrid namespace clients, client groups, topic spaces, and permission bindings is offered:

From there, we can connect all relationships:

• Each client is part of one or more client groups based on their queries.
• Each client group can be used in one permission binding.
• Each permission binding relates to a topic space and defines the permissions.
• Each topic space has one or more topic templates.

This seems simple, doesn’t it?

There was one problem…

The library in the NuGet packages does not support executing the query so we get a list of clients with the right client attributes set.

Because I did not want to wait until this query support was added, I tried to create my own query parser.

Custom client group query parser

You have to know that writing a query parser is hard!

I looked up the query documentation and the query is quite intelligent:

It can even cope with ‘and’, ‘or’, and parenthesis!

That is why my query parser only supports a subset of the query capabilities:

• Equality operator “=”
• Not equal operator in two forms “<>” and “!=”
• “IN” to compare with a set of values

Here are some examples:

attributes.type IN ['audit']
attributes.type IN ['audit', 'a', 'AA',99]
attributes.type in ['audit']
attributes.type in ['audit', 'a', 'AA',99]
attributes.type = "audit"
attributes.type != "audit"
attributes.type <> "audit"

So, I also added a ‘DeviceClientQueryTestProject’ unit test project:

You are welcome to contribute with a more elaborate query parser as long as it comes up with successful unit tests.

Conclusion

This was a fun project with a great outcome.

It shows you how to list all topics and permissions for each client in your EventGrid namespace-supported MQTT broker.

The library uses a ResourceManager EventGrid NuGet package so you can see how it’s done and start building your own logic.

The custom query parser does not support the full set of query capabilities but it should work in most of the use cases.

This Nuget package also seems useful for manipulating an EventGrid Namespace but is something for another occasion.