Use the MQTT Broker Like a Pro + Examples

Description

This tutorial is for people who have purchased a RAK WisGate Edge gateway and want to use the Built-in Network Server functionality to its fullest.

Here the user will learn how to subscribe to a topic to obtain data from your end-devices through MQTT. For convenience, the Built-in Network Server will be used in the examples.

This article is suitable for all RAK WisGate Edge series gateways (opens new window).

As for the examples, we will introduce how to use the built-in MQTT as well as an external one.

#A Brief Introduction to LoRaWAN and MQTT

If you are familiar with the two, you can skip this part.

#LoRaWAN

As shown in Figure 1, LoRaWAN network roles are divided into:

End nodes - The end-devices responsible for data collection. the data is then encrypted and transmitted to de gateway in the form of a wireless signal.
Concentrator/Gateway - They collect the data from multiple nodes and send it to the Network server (NS).
Network Server - The NS decrypts and processes the data forwarded by the gateway according to the data identification information and key.

Figure 1: LoRaWAN network architecture

RAK simplifies the actual deployment of a LoRaWAN network by integrating an NS in all our commercial gateways.

#MQTT

MQTT stands for Message Queuing Telemetry Transport, which is a publish/subscribe, extremely simple and lightweight, messaging protocol, intended for use on constrained devices and low-bandwidth, high-latency, or unreliable networks.

For more information, check out our article about MQTT (opens new window).

Figure 2: MQTT architecture

#The Role of MQTT in the LoRaWAN Networks

Practical application 1: Use the built-in MQTT Broker of RAK commercial gateway

The WisGate Edge gateway obtains data from the nodes and sends it to the built-in NS, it then publishes the data to the built-in MQTT Broker. In this case, the gateway is both the publisher and the broker at the same time. The user then subscribes to the topic through third-party programs, like MQTT.fx, to be able to manage that data.

Practical application 2: Use the built-in NS to publish data to an external MQTT Broker

The RAK commercial gateway obtains the data and sends it to the built-in NS, and it then publishes the data to a third-party MQTT Broker. Users subscribe through third-party programs. In this case, the gateway is only a publisher.

#Configuring the Built-in Network Server

This tutorial is for the WisGate Edge series of devices, all of which come pre configured in Network Server operation mode, so there is no need to adjust this setting unless you have changed it.

For details on configuring the settings mentioned above visit the WisGate Edge Documentation(opens new window)

However, there might be some adjustments needed for a particular setup like this one. Depending on your location, you have to make sure you have set the channel plan to one corresponding to your regional frequency band. Depending on the gateway variant, the default channel plan might not be the one you need.

On how to make changes, if needed, watch our instructional video (opens new window), or read this guide (opens new window).

#Use the Built-in MQTT Broker

This chapter will show you how to subscribe to topics, receive and send data reported by nodes through MQTT.fx, using the build-in MQTT Broker. You will need the MQTT.fx tool (MQTT clients) that can be downloaded here (opens new window).

#Create an Uplink Topic

The data is transmitted by using an Uplink Topic. To see the MQTT topic templates for your gateway, in the Web UI, navigate to LoRa Network > Global Integration > Uplink Topic.

Figure 3: Global Integration topic templates

📝 NOTE

Do not change the topics in your gateway. They are just templates of what your topic should look like and what the built-in MQTT Broker uses.

To generate a correct Uplink Topic you will need the template and the following information:

application_ID – This is the ID of your application. To see it, navigate to LoRa Network > Application.

Figure 4: Applications list

device_EUI – This is the EUI of your device. To see it, navigate to LoRa Network > Application > Edit.

Figure 5: Devices list

With this information, the topic template application//device//rx will change to something like application/1/device/60c5a8fffeXXXXXX/rx.
If you want to subscribe to all nodes in an application, use the following topic (replace with the ID of your application):

application/{{application_ID}}/device/+/rx
 
        Copied!

If you want to subscribe to data from all applications, use the topic:

application/+/device/+/rx
 
        Copied!

Open the MQTT.fx tool and click on the New button ().

Figure 6: MQTT.fx startup window

Type the IP address of the gateway and click Connect. The black indicator on the far right should turn green, indicating that the connection is successful.

Figure 7: Connecting MQTT.fx

📝 NOTE

For this tutorial, the gateway is connected to the local network, so the IP is 192.168.0.102. Yours will be different.

To connect to the topic, navigate to the Subscribe tab, type in the Uplink Topic you created, and click the Subscribe button.

Figure 8: Subscribing to the topic

You can send a message from the node to test the connection. Via the RAK Serial Port Tool (opens new window), you can send a “Hello RAKwireless” message. The data must be in HEX, so if you want to send the message, use the following command:

at+send=lora:1:48656c6c6f2052414b576972656c657373
 
        Copied!

📝 NOTE

To convert any message in HEX, use a converter like this (opens new window).

Figure 9: RAK Serial Port Tool

📝 NOTE

In this example, RAK7204 WisNode Sense Home is used. However, the tutorial will work with any LoRaWAN device.

In the MQTT.fx interface, you can see the message you just sent. The message is encoded in base64 format.

Figure 10: Uplink data

#Send Data to the Node via MQTT.fx

To send data to the nodes (downlink) requires publishing the data to the Downlink Topic. To see the topics templates, go to the Create an Uplink Topic part for reference. Find the Downlink Topic template and replace the application_ID with the ID of your application and device_EUI with the EUI of your device.

An example of Downlink Topic would look something like application/1/device/60c5a8fffeXXXXXX/tx.

📝 NOTE

The Downlink Topic ends with tx.

In the MQTT.fx navigate to Publish tab and type the name of your topic.

Figure 11: Entering Downlink Topic

To send a message to the node, in the field under the topic’s name type the command for it and click the Publish button. To send a "Hello" message, use the command:

{"confirmed":true,"data":"SGVsbG8=","fPort":10}
 
        Copied!

Where:

Confirmed can be true or false.
The content of the data is the information you want to send. The information must be base64 encoded. This can be done with a converter (opens new window).
fPort is the port number where you send the data. The number can be 1-255.

Figure 12: Publish a message

The message you just sent can be seen in the RAK Serial Port tool and it is encoded in HEX format.

Figure 13: Received message

📝 NOTE

If the device is configured to work in Class C, it will receive the message immediately after sending. If the node is working in Class A, it will receive the message on the next Uplink.

#Use an external MQTT Broker

This section will explain how to configure the gateway to use an external MQTT Broker and how to connect MQTT.fx to it.

📝 NOTE

This section describes how to access the external MQTT by username/password.

#Configuring the Gateway and MQTT.fx

To configure the gateway to point to an external MQTT server, in the Web UI, navigate to LoRa Network > Global Integration and type the MQTT Broker Address of the server you are going to use. Switch the Enable User Authentication, fill in the Username and Password to access the MQTT server, and click Save & Apply.

Figure 14: Configuring the gateway

📝 NOTE

Enable the User Authentication only if the MQTT Broker requires Username and Password for authentication.

Open the MQTT.fx and click on the Setting icon

Figure 15: MQTT.fx startup window

In the Edit Connection Profiles click on the plus icon ( ) in the lower-left corner to create a new Profile. Type in Profile name of your choice and the IP of the Broker Address. Click on User Credentials to enter the User Name and Password to access the MQTT server and then click OK.

Figure 16: Edit Connection Profiles

From the drop-down menu (left of the Connect button) at the main window of the MQTT.fx choose the Profile you just created and click Connect to connect to the external MQTT server. The black dot on the far right should turn green, indicating that the connection is successful.

Figure 17: Connecting to the new profile

Figure 18: Successful connection

To subscribe, or to publish data, refer to the corresponding sections from the previous use case: Subscribe to the Topic via MQTT.fx and Send Data to the Node via MQTT.fx.

#Format Definition of the Obtained Through MQTT Data

In the previous sections, you were shown how to subscribe to Uplink data and how to send a Downlink to the node. But there are three more types of data:

Join – This is the information received when the node is connected to the network.
Ack – This is the confirmation message that the node sends as a reply after receiving a Downlink.
Status – This is information about the node’s battery level.

#Uplink

{
	"applicationID": "1",		// The ID of the application in which the node is registered
	"applicationName": "RAK_mqtt_test",		// The name of the application
	"devEUI": "60c5a8fffeXXXXXX",		// The EUI of your device
	"deviceName": "RAK7204",		// The name of your device
	"timestamp": 1618827808,		// Unix timestamp of received node data
	"fCnt": 30,		// The frame counter of the message
	"fPort": 8,		// The port to which the data is send
	"data": "CAIBaQdoTAZzJbcCZwDyBAINRQ==",		// The data sent by the node
	   "data_encode": "base64",		// Decoding data type
 	   "adr": true,		// Whether the node has adaptive data rate enabled
 	   "rxInfo": [	// Information about the gateway, that received the node data
 	       {
  	          "gatewayID": "f6e529fffeXXXXXX",		// Gateway’s EUI
   	         "loRaSNR": 9.8,		//Signal-to-noise ration of the message
   	         "rssi": -41,		// Received Signal Strength Indicator of the message
    	        "location": {		// GPS information of the gateway
          		      "latitude": 42.88595,
          		      "longitude": 25.31158,
          		      "altitude": 437
        	    },
         	   "time": "2021-04-19T10:23:28.126943Z"
   	     }
	    ],
	    "txInfo": {
 	       "frequency": 867900000,		// The frequency used by the node to send data
  	      "dr": 5		// Node’s data rate
 	   }
}
 
        Copied!

#Downlink

{
	"confirmed": true,		// Show is the message confirmed or not
	"fPort": 10,		// The port used for the Downlink message
	"data": "SGVsbG8gUkFLV2lyZWxlc3M="		// The data sent to the node, encoded in base64 format
}
 
        Copied!

#Join

{
	"applicationID": "1",		// The ID of your application
	"applicationName": "RAK_mqtt_test",		// The name of your application
	"deviceName": "RAK7204",		// The name of your device
	"devEUI": "60c5a8fffeXXXXXX",		// The EUI of your device
	"devAddr": "0282e506"		// Short address assigned to your device after a successful join
}
 
        Copied!

#Ack

{
	"applicationID": "1",		// The ID of the application in which the node is registered 
	"applicationName": "RAK_mqtt_test",		// The name of the application
	"deviceName": "RAK7204",		// The name of your device	
	"devEUI": "60c5a8fffeXXXXXX",		// The EUI of your device
	"acknowledged": true,		//Acknowledgement confirmation
	"fCnt": 2		// The frame counter of the message
}
 
        Copied!

📝 NOTE

The node will reply with Ack only after the server has sent the confirmed type of data to the node. The node may not reply to the Ack immediately. The Ack may be carried in the next uplink data sent by the node.

#Status

{
	"applicationID": "1", 
	"applicationName": "RAK_mqtt_test",
	"deviceName": "RAK7204",
	"devEUI": "60c5a8fffeXXXXXX",
	"battery": 254,		// The classification of the remaining battery power. 254 means the power supply is fully charged, 1 means battery is about to run out
	"margin": 8,		// It is the demodulation signal-to-noise ratio of the last successful reception of the DevStatusReq command
	"external PowerSource": false,		// Whether additional power source is presented
	"batteryLevel": 100		//When "external PowerSource" is true, batteryLevel represents the percentage of battery
}
 
        Copied!

#Example Program

In this section, a python code will be shown, that calls the MQTT subscription node's uploaded data and prints the corresponding content. When receiving Uplink data, the program will send a “Hello RAKwireless” message to the node. Make sure you read the code comments carefully before coding.

📝 NOTE

For this tutorial, the example program is running on a Raspberry Pi with installed Ubuntu and it is working for both practical applications. The needed commands can be run on a terminal via SSH, or directly in the graphical environment of Ubuntu.

For this tutorial, we will use an external MQTT Broker that is installed on a Raspberry Pi. If you want to build an MQTT server on a Raspberry Pi, run the following commands:

sudo apt install mosquitto mosquitto-clients
 
        Copied!

sudo systemctl enable mosquitto
 
        Copied!

You can check the status of the MQTT Broker by using the command:

sudo systemctl status mosquitto
 
        Copied!

Python dependencies need to be installed. This is done with the commands:

sudo apt install python-pip
 
        Copied!

pip install paho-mqtt
 
        Copied!

Create a new file where you will put the code, using the command:

sudo nano yourfile.py
 
        Copied!

📝 NOTE

Make sure when creating the file, you save it as a python file with the correct extension (yourfile**.py**).

The content of the file is this code (make sure you read the explanations and make the needed changes for your particular case):

#!/usr/bin/env python

import json
import base64
import paho.mqtt.client as mqtt
from datetime import datetime

# Type the IP of your server
mqtt_ip = 'ip of your MQTT server' 
mqtt_port = 1883
# Type your username and password. If the private MQTT server does not require username and password, commend the lines
mqtt_username = 'username'
mqtt_password = 'password'
# Replace the "id" with the id of your application and "eui" with your device eui
mqtt_rx_topic = 'application/id/device/eui/rx'

# Convert string to hexadecimal
def str_to_hex(s):
	return r"\x"+r'\x'.join([hex(ord(c)).replace('0x', '') for c in s])
# Once subscribed to the message, call back this method
def on_message(mqttc, obj, msg):
	on_print_rak_node_info(msg.payload)
# Print the subscribed node information
def on_print_node_rx_info(json_rx):
	try:
		devEUI = json_rx['devEUI']
		applicationID = json_rx['applicationID']
		applicationName = json_rx['applicationName']
		deviceName = json_rx['deviceName']
		timestamp = json_rx['timestamp']
		fCnt = json_rx['fCnt']
		fPort = json_rx['fPort']
		data = json_rx['data']
		data_hex = str_to_hex(base64.b64decode(data).decode("latin-1"))
		# Convert the timestamp to local time
		str_local_time = datetime.fromtimestamp(timestamp)
		print('---------------- devEUI:[%s] rxpk info -------------------' % devEUI)
		print('+\t applicationName:\t%s' % applicationName)
		print('+\t applicationID:\t\t%s' % applicationID)
		print('+\t deviceName:\t\t%s' % deviceName)
		print('+\t datetime:\t\t%s' % str_local_time)
		print('+\t fCnt:\t\t\t%d' % fCnt)
		print('+\t fPort:\t\t\t%d' % fPort)
		print('+\t data:\t\t\t%s' % data)
		print('+\t data_hex:\t\t%s' % data_hex)
		print('----------------------------------------------------------')
	
	except Exception as e:
		print(e)
	finally:
		pass
# After subscribing to the node's data, send the "Hello RAKwireless" string to the node
def on_print_rak_node_info(payload):
	json_str = payload.decode()
	try:
		json_rx = json.loads(json_str)
		on_print_node_rx_info(json_rx)
		dev_eui = json_rx['devEUI']
		app_id = json_rx['applicationID']
# Industrial gateway default tx topic
		tx_topic = 'application/%s/device/%s/tx' % (app_id, dev_eui)
		str_hello = "Hello RAKwireless"
		tx_msg = '{"confirmed":true,"fPort":10,"data":"%s" }' % str(base64.b64encode(str_hello.encode("utf-8")).encode("utf-8"))
		mqttc.publish(tx_topic, tx_msg, qos=0, retain=False)
		print('Send \'Hello RAKwireless\' to node %s' % dev_eui)
	
	except Exception as e:
		raise e
	finally:
		pass
	
mqttc = mqtt.Client()
mqttc.on_message = on_message
# If there is no username and password, please comment the following line:
mqttc.username_pw_set(mqtt_username, password=mqtt_password)
# Connect to mqtt broker, the heartbeat time is 60s
mqttc.connect(mqtt_ip, mqtt_port, 60)
mqttc.subscribe(mqtt_rx_topic, 0)
mqttc.loop_forever()
 
        Copied!

Run the code by typing in the SSH terminal:

python yourfile.py

in McCoy IoT

# LoRaWAN MQTT

Description

#A Brief Introduction to LoRaWAN and MQTT

#LoRaWAN

#MQTT

#The Role of MQTT in the LoRaWAN Networks

#Configuring the Built-in Network Server

#Use the Built-in MQTT Broker

#Create an Uplink Topic

#Send Data to the Node via MQTT.fx

#Use an external MQTT Broker

#Configuring the Gateway and MQTT.fx

#Format Definition of the Obtained Through MQTT Data

#Uplink

#Downlink

#Join

#Ack

#Status

#Example Program

RELATED PRODUCTS:

RAK7249 WisGate Edge Max

RAK7258 WisGate Edge Lite

RAK7268 WisGate Edge Lite 2

RAK7289 WisGate Edge Pro

Follow us

Description

#A Brief Introduction to LoRaWAN and MQTT

#LoRaWAN

#MQTT

#The Role of MQTT in the LoRaWAN Networks

#Configuring the Built-in Network Server

#Use the Built-in MQTT Broker

#Create an Uplink Topic

#Subscribe to the Topic via MQTT.fx

#Send Data to the Node via MQTT.fx

#Use an external MQTT Broker

#Configuring the Gateway and MQTT.fx

#Format Definition of the Obtained Through MQTT Data

#Uplink

#Downlink

#Join

#Ack

#Status

#Example Program

RELATED PRODUCTS:

RAK7249 WisGate Edge Max

RAK7258 WisGate Edge Lite

RAK7268 WisGate Edge Lite 2

RAK7289 WisGate Edge Pro