Somewhere in December 2015, I was made aware of this Lora initiative called The Things Network. Since then, as an IoT enthusiast, I am researching how to implement this platform in my other IoT projects.

Update 8-11-2016: this blog gives an introduction to MQTT and accessing the TTN network. A full implementation of a C# TTN->Azure bridge is available at GitHub. More details are available here.

Lora stands for Long Range and it fills a gap between Wifi and GMS, thinking about wireless connectivity for IoT sensor boards:

lora2

At the moment there are two serious implementations of Lora in The Netherlands. KPN is offering a commercial solution so it’s reliable but it does not come free. And then there’s The Things Network, a Kickstarter solution. It offers free connectivity :-).

TTN-Overview

What they ‘sell’ are gateways. These come fairly cheap (starting at ~ 250 euros) but with their gateway, you can connect up to 5 kilometers (keep on dreaming about 10 :-)) around your house with Arduino’s, ESPs, RaspberryPi etc. And with a couple of these gateways, you can cover your village or city. So get your friend involved!

The telemetry of the nodes (say twenty bytes of date every minute) are received by the gateways and forwarded to the TTN backend. But you have to do ‘something’ yourself to get the data from the backend. I will tell you how to do that using C# all the way.

I presume you already have some knowledge about the TTN communication before you can proceed here 🙂

This is the near future coverage of the city of Eindhoven (not all circles are already live):

map-eindhoven

A more reliable impression comes from ttnmapper.org:

ttnmapper

In April 2016, I laid my hands on a beta node of TTN during the IotTechdday.nl. And a few weeks later I received a package forwarder which can mimic a gateway.

So in may 2016, I and a few colleagues were able to send telemetry to the backend using our own nodes and package forwarders.

You can see your data arriving at the backend when you drill down in your application settings and after that in the device settings:

WhatsApp-Image-20160708

Note: the data for this blog is coming from a node which is build by my dear colleague Hans Boksem.

Getting data from the backend, the TTN way

During workshops given by the TTN, telemetry is normally taken from the TTN backend, and represented,  using NodeRed:

node-red-screenshot-sm

NodeRed is an excellent environment. It’s based on NodeJs and it used a graphical interface to allow users, even non-technical ones, to build fairly complex flow structures with only a few building blocks.

But then again, I never liked it. First of all (it’s a bit personal) it isn’t a .Net solution. I would like to see the telemetry handled by c# code in Azure. And secondly, it’s a very heavy solution for just getting my hands on my own data. Why? All I need is something that talks against the TTN backend using the MQTT protocol. Because the protocol is supported by TTN.

MQTT

“MQTT is an ISO standard publish-subscribe-based “lightweight” messaging protocol for use on top of the TCP/IP protocol” (Wikipedia)

So I looked around in the .Net universe and found M2Mqtt:

“M2Mqtt is an MQTT client available for all .Net platform (.Net Framework, .Net Compact Framework and .Net Micro Framework) and WinRT platform (Windows 8.1 and Windows Phone 8.1) for M2M communication.”

Great, there’s even a Nuget package.

Connecting with MQTT using C#

So I finally came up with this console app (some transformation classes will be provided below):

internal class Program
{
  private static void Main(string[] args)
  {
    try
    {
      var client = new MqttClient(
            Settings.Default.BrokerHostName);

      client.MqttMsgPublishReceived +=
        Client_MqttMsgPublishReceived;

      var clientId = Guid.NewGuid().ToString();

      var subscriptionId = client.Subscribe(
        new string[] { Settings.Default.Topic },
        new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });

      client.ConnectionClosed += Client_ConnectionClosed;

      client.MqttMsgSubscribed += Client_MqttMsgSubscribed;

      var connectionId = client.Connect(
            clientId,
            Settings.Default.Username,
            Settings.Default.Password);
    }
    catch (Exception ex)
    {
      Console.WriteLine("Exception" + ex.Message);
    }

    Console.WriteLine("Press a key to exit...");
    Console.ReadKey();
  }

  private static LoraTelemetry GetTelemetry(string payload)
  {
    var text = payload
                .Replace("%", string.Empty)
                .Replace("C", string.Empty);

    var values = text.Split(',');

    return new LoraTelemetry
    {
      Humidiy = string.IsNullOrEmpty(values[0])
           ? 0 : Convert.ToDecimal(values[0]),
      TemperatureCelcius = string.IsNullOrEmpty(values[1])
           ? 0 : Convert.ToDecimal(values[1]),
      TemperatureCelciusExt = string.IsNullOrEmpty(values[2])
           ? 0 : Convert.ToDecimal(values[2]),
    };
  }

  private static void Client_MqttMsgPublishReceived(
                  object sender, MqttMsgPublishEventArgs e)
  {
    var jsonText = Encoding.ASCII.GetString(e.Message);

    var message = Newtonsoft.Json.JsonConvert.
             DeserializeObject<loramessage>(jsonText);

    var payload = Base64Decode(message.payload);

    var telemetry = GetTelemetry(payload);

    Console.WriteLine($"D {message.dev_eui}
                        H {telemetry.Humidiy}
                        C {telemetry.TemperatureCelcius}
                        C2 {telemetry.TemperatureCelciusExt}");
  }

  public static string Base64Decode(string base64EncodedData)
  {
   var base64EncodedBytes =
          Convert.FromBase64String(base64EncodedData);
   return Encoding.UTF8.GetString(base64EncodedBytes);
  }

  private static void Client_MqttMsgSubscribed(
       object sender, MqttMsgSubscribedEventArgs e)
  {
    Console.WriteLine("Client_MqttMsgSubscribed: " + e.ToString());
  }

  private static void Client_ConnectionClosed(object sender, EventArgs e)
  {
    Console.WriteLine("Client_ConnectionClosed: " + e.ToString());
  }
}

To get this code running we still need some extra settings:

lorasettings

First of all, you have to provide the staging ‘end point’, the broker host name.

After that, you will need to provide a name and password. This is just the combination of the App EUI and the accompanying session key proving you are the owner of this telemetry.

Finally, you will have to provide the name of the topic you are interested in. I wanted to get all date of just one device so I passed the topic name of that device (like “AppEUI/devices/00000000[DeviceEUI]/up”). But if you are interested in telemetry of ALL devices, just pass a “#” (conform MQTT specs).

Now just connect and see the telemetry coming in, in the Client_MqttMsgPublishReceived event.

Some transformation needed

So what you receive (after some transformation stuff) is a JSON message:

{"payload":"NDcuMiUsMjYuMkMs",
"port":1,
"counter":386,
"dev_eui":"00000000452A2912",
"metadata":[{"frequency":868.3,
"datarate":"SF7BW125",
"codingrate":"4/5",
"gateway_timestamp":3959544198,
"channel":1,
"server_time":"2016-07-07T17:21:52.175318158Z",
"rssi":-65,
"lsnr":9,
"rfchain":1,
"crc":1,
"modulation":"LORA",
"gateway_eui":"B827EBFFFFA3A2A3",
"altitude":0,
"longitude":6.62672,
"latitude":52.41504}]}
}

It’s interesting to see which gateway picked up your message and what the reception was (RSSI) during the transmission. The location (latitude/longitude) is just provided by the owner of the gateway. There is no GPS involved.

We can transform this JSON string into a C# object:

public class loramessage
{
  public string payload { get; set; }
  public int port { get; set; }
  public int counter { get; set; }
  public string dev_eui { get; set; }
  public metadataitem[] metadata { get; set; }

  public class metadataitem
  {
    public decimal frequency { get; set; }
    public string datarate { get; set; }
    public string codingrate { get; set; }
    public Int64 gateway_timestamp { get; set; }
    public int channel { get; set; }
    public string server_time { get; set; }
    public int rssi { get; set; }
    public int lsnr { get; set; }
    public int rfchain { get; set; }
    public int crc { get; set; }
    public string modulation { get; set; }
    public string gateway_eui { get; set; }
    public decimal altitude { get; set; }
    public decimal longitude { get; set; }
    public decimal latitude { get; set; }
  }
}

But then, we still do not own the actual telemetry, the payload. This is just a Base64 encoded text in this example so it gives:

NDcuMiUsMjYuMkMs decodes into 47.2%,26.2C,

So we can decode it and put it in another class:

public class LoraTelemetry
{
  public decimal Humidiy { get; set; }
  public decimal TemperatureCelcius { get; set; }
  public decimal TemperatureCelciusExt { get; set; }
}

And there you have it: Lora messages in a DOSBox extracted by a console application:

telemetry

That was not so hard while NodeRed can be fairly impressive and even overwhelming…

Observations

There are some observations to make.

It’s good to know that multiple listeners can read the same telemetry at the same time. And listeners have to be kept in memory, they are stateful, they open multiple ports to communicate with the server (conform MQTT specs).

It should be possible to let an M2Mqtt register for MULTIPLE topics, multiple devices. We know which devices exist so we can balance the load of the telemetry over multiple MQTT clients. Unfortunately, I did not get this working at the moment of writing this blog.

Then I’m still a bit concerned about the scalability. What if I have to read telemetry from, say, a million devices. Then I must prevent the situation where the same message from the same device is read by multiple clients. This will need some administration.

Conclusion

This solution makes way for putting the code in Azure, in front of an IoTHub.

I already have it running in a WorkerRole. Although it’s a bit outdated and deployment is slow, it’s a solution which supports long running processes (keep in mind, the client has to be kept instantiated).

I will look into WebJobs too but I’m afraid this is a more expensive solution (I’m using for this experiment the one-year free 25 Euros Azure subscription, part of the Developer Program Benefits ) than my ExtraSmall WorkerRole.

Update 8-11-2016: this blog gives an introduction to MQTT and accessing the TTN network. A full implementation of a C# TTN->Azure bridge is available at GitHub. More details are available here.

Advertenties

One thought on “Access The Things Network Lora telemetry using C# M2Mqtt

Reacties zijn gesloten.