Forget about messages, just care about IoTHub chunks

Ok, this blog is all about taking the blue pill or the red pill:

I use the Azure IoTHub for every project I am involved in. It’s my main focus on getting IoT things done and using this tool is how I manage.

And consequently, I always look at the number of daily messages we have to ingest because the price of an Azure IoTHub depends on the number of messages it passes on.

It starts with 400.000 messages a day and from there, you can scale up. The biggest version can scale up to a tenfold of 300 million messages a day:

But this is not the whole truth, you can send more…

First, let’s define what a message is.

Messages

For me, a message is a piece of JSON which contains some values collected at a certain point in time by a unique device.

This message can get some context in the form of properties which can be used for routing. For example, I can distinct raw telemetry messages from alarm messages and heartbeat messages. This is how I route my messages.

And every message I sent to the IoTHub is counted:

This week I was checking the messages too and I noticed the number of messages, almost 65 thousand at that moment. And this number of messages was quite high for the time of the day. So I checked the database which collects all messages as records and I only found 40 thousand or so messages in the database.

Luckily, the IoTHub has some nice metrics so I checked them out too:

Wut?

Why is the number of messages charged for, higher than the number of actual messages received?

Chunks

The explanation is quite simple, you have to check the size of the messages!

The maximum size of Device-to-cloud messages is 256KB.

And that’s OK. In our project, we collect numerous messages and a few of them are aggregations of all raw data we get from machines each minute. So these messages are more average sized messages.

And that’s the difference: messages are metered in chunks of four KB. So, if your message is ten KB big, you will be metered in three chunks. If your message is ten bytes big or a thousand bytes or even three thousand KB, you will still be metered for only one chunk.

So try to limit the number of chunks!

For Microsoft, a chunk is something that contains one or more messages, if they are smaller than a chunk. So, if you are not in a hurry, try to put multiple messages in a chunk and sent that one over the line to the IoTHub.

We are now in the process of checking if we can aggregate raw data in chunks of four KB. We want to know if we can get our chucks filled to the max before we sent them. This is not easy, but Microsoft is transparent on how the size of messages is calculated.

Files

We are also sending files to the cloud using Azure IoT.

The files were a bit too large for the maximum message size limitation but we managed to zip them. We can still receive and unzip them in Azure Functions after being routed by the IoTHub.

So if we send a file, zipped to a message of 200KB, it is metered in fifty chunks.

Microsoft also supports sending files to the Cloud using the File upload option of the IoTHub. So you use the device credentials to store files in a related blob storage.

According to the documentation, every file, no matter how big it is, is only charged for two chunks and the Azure storage costs for that file.

That is a big difference is you send many files each day.

But in our case, we only received one or two files a day so we still prefer the power of the routing and uniformity.

Conclusion

Using the cloud gives you unlimited compute, elastic behavior of resources and real-time monitoring of usage and costs of resources.

That’s how deep this Rabbit Hole goes 🙂

The same goes for the IoT platform in general and the IoTHub specifically. You have to get control over the number of chunks.

Use this resource smart and keep an eye on the size of your messages.