MQTT Client Protocol (Message Queue Telemetry Transport)

Supported device types and versions 
About the MQTT Sparkplug protocol
Communication line configuration
Communication line parameters
Communication station configuration
I/O tag configuration
Tell commands
Literature
Document revisions

Supported device types and versions

The protocol is an implementation of the MQTT 3.1.1 standard (October 2014) and the MQTT 5.0 standard (March 2019). MQTT protocol is a client/server protocol of a subscribe/publish type. It is simple, has little overhead, and is easy to implement. It is used for M2M communication (Machine to Machine) and in the IoT context (Internet of Things). The MQTT server is also called the MQTT broker.
D2000 KOM implements the client part of the protocol. The protocol is implemented on TCP/IP-TCP and TCP/IP-TCP Redundant lines. MQTTS (Secure MQTT, MQTT over TLS) is also supported - either directly in the D2000 KOM process (using TLS settings on TCP/IP-TCP and TCP/IP-TCP Redundant lines) or via the stunnel utility.
For the transfer of LoRaWAN data encapsulated within the MQTT protocol, see LoRaWan protocol description.

The Sparkplug 3.0.0 standard (Sparkplug B MQTT Payload) is also supported. Support includes:

• Connecting to the MQTT broker as a Host Application (data consumer, in Payload Type = Sparkplug Host mode):

  • reading simple value types

  • reading arrays (with support for the Destination column)

  • writing simple value types

  • reading of template items (UDT)

  • reading of dataset items

  • support for metric aliases

  • compression support (GZIP, DEFLATE)

  • browsing - finding a list of metrics

• Connecting to an MQTT broker as an Edge Node (data producer, in Payload Type = Sparkplug Edge Node mode):

  • publishing values ​​with simple types

  • processing commands (NCMD, DCMD) with simple value types

  • processing commands (NCMD, DCMD) with template (UDT) items

  • processing commands (NCMD, DCMD) with dataset items

  • processing commands (NCMD, DCMD) with arrays

  • processing of the NCMD command with the Node Control/Next Server metric (connecting to the next MQTT server, if several are configured on the line)

  • processing of the STATE message with the content online=false, which comes from the Primary Host Application (disconnecting from the MQTT server and trying another one, if several are configured on the line - searching for an MQTT server with a connected Primary Host Application)

  • support for metric aliases

The communication was tested/deployed with:

• www.TheThings.network cloud 

• Loriot.io cloud 

• Slovanet cloud (loralink.slovanet.sk)

• Pixii PowerShaper (energy storage solution)

• liveobjects.orange-business.com cloud

• test.mosquitto.org (in both Payload Type = Sparkplug Host and Payload Type = Sparkplug Edge Node modes)

• broker.hivemq.com (in both Payload Type = Sparkplug Host and Payload Type = Sparkplug Edge Node modes)

• ThingsBoard (in Payload Type = Sparkplug Edge Node mode)

• Eclipse™ Sparkplug™ Technology Compatibility Kit (TCK) using version 3.0.0

Note: communication with the cloud liveobjects.orange-business.com via WebSockets (wss://liveobjects.orange-business.com:443/mqtt) was also tested. The program https://github.com/jimparis/unwebsockify.git was used as a WSS wrapper. This program started with the following parameters:
./unwebsockify.py --port 1883 --listen 172.16.0.1 wss://liveobjects.orange-business.com:443/mqtt
The D2000 KOM process was connected to address 172.16.0.1 on port 1883. The WSS wrapper was connected to the defined URL and wrapped the MQTT communication data in a WebSocket envelope.

Each PUBLISH message contains a topic (Topic), data (Payload), and level of confirmation (QoS). PUBLISH messages can be sent both by the client and the server. The clients at the beginning of the communication will use the SUBSCRIBE message to indicate what topics (parameter of Topic Filter protocol) they are interested in. 

• QoS_0 - PUBLISH message is not confirmed; it may be lost

• QoS_1 - PUBLISH message is confirmed by the other side's PUBACK, it may be duplicated

• QoS_2 - PUBLISH message is confirmed by the other side's PUBREC, which is then confirmed back by the PUBREL message, and that one by a final PUBCOMP message.

The level of confirmation of the messages sent by the D2000 KOM process is defined by the protocol parameter Publish QoS. The D2000 KOM process considers the writing of the output tag to be successfully finished depending on the QoS:

• QoS_0 - after the data is successfully sent via the TCP connection

• QoS_1 - after receiving PUBACK

• QoS_2 - after receiving PUBCOMP

The MQTT communication starts with the CONNECT message sent by the client (D2000 KOM). The message contains User Name, Password, and other parameters, which can be modified:  Clean Session Flag, Client ID, Will Flag, Will QoS, Will Retain, Keep Alive, and Will Topic. The server replies with a CONNACK message with a return code that contains information about the success of the connect operation.

Then the client sends a SUBSCRIBE message with a filter of topics (Topic Filter parameter), specifying which topics it is interested in, and with the required maximum level of confirmation (parameter Subscribe QoS).

The server responds with a return code that contains information about the success and the maximum QoS that was assigned to the requested topics.

Then follows a phase of communication, during which both the client and the server send PUBLISH messages (the client with any topic, the server with topics relating to the filter of topics of the received SUBSCRIBE message) and confirm them according to the value of the QoS parameter of the received PUBLISH messages.

If the server does not send a message for longer than the Ping Interval seconds, the client sends the PING request message, to which the server must respond with the PING response message (within the time specified by the Reply Timeout parameter).
If parameters change on the line, the connection is closed and re-created.

About the MQTT Sparkplug protocol

Sparkplug's MQTT protocol defines 3 groups of applications:

• Edge Node - Supports the MQTT protocol and connects to the MQTT server. It sends him data obtained from the Device, its own, or aggregated data.

• Device/Sensor - Represents a physical or logical device connected to an Edge Node and providing process data or metrics.

• Host Application - Represents a data consumer (SCADA/MES system, Historian, analysis tool) that connects to the MQTT server and receives MQTT data from the Edge Node/Device and, if necessary, commands.

The D2000 can operate in Host Application mode or Edge Node mode (plus one or more Device/Sensor devices) according to the Payload Type line parameter. In Edge Node mode, exactly one station with an Edge Node address and optionally additional stations with slave Device/Sensor addresses must be configured on the line.

Open

MQTT Sparkplug defines the following message types:

• NBIRTH – Birth certificate for Sparkplug Edge Nodes (information about the connection of the Edge Node to the MQTT Server)

• NDEATH – Death certificate for Sparkplug Edge Nodes (information about the loss of the connection between the MQTT Server and the Edge Node)

• DBIRTH – Birth certificate for Devices (information about Device connection to Edge Node)

• DDEATH – Death certificate for Devices (information about the loss of Edge Node connection with Device)

• NDATA – Edge Node data message (data from Edge Node)

• DDATA – Device data message (data from Device)

• NCMD – Edge Node command message (Host Application command for Edge Node)

• DCMD – Device command message (Host Application command for Device)

• STATE – Sparkplug Host Application state message (information from the state of the Host Application for the Edge Node/Device [online/offline]).

MQTT Sparkplug defines a Topic syntax for all message types, which has the form namespace/group_id/message_type/edge_node_id/[device_id], where:

• namespace is a spBv1.0 constant (for Sparkplug B)

• group_id is an arbitrary name of a logical group (e.g. by device type, operation, organizational structure, etc.)

• message_type is the message type

• edge_node_id is the identifier of the Edge Node

• device_id is the identifier of the Device (only if the message is from/for a Device)

Examples:

• spBv1.0/Sparkplug Devices/DDATA/Marvin/Sensor1

• pBv1.0/MIMIC/DDATA/20:19:AB:F4:0E:0D/sensor11

• spBv1.0/SparkplugDevices/NBIRTH/JsonScada

Note: STATE messages have a Topic of the form spBv1.0/STATE/sparkplug_host_id, where sparkplug_host_id is the unique name of the host application  (Sparkplug Host ID parameter).

Examples:

• spBv1.0/STATE/CFGHUB2023

• spBv1.0/STATE/D2Kom

All message types except STATE have a binary Sparkplug payload. The STATE messages have a JSON payload - they contain only the items "online" (True/False value) and "timestamp" (time stamp in milliseconds).

The Sparkplug payload contains metrics - each required metric can be configured into a single I/O tag. The metric contains a text identifier (specified in the I/O tag configuration as sparkplug_address), value type, value (simple/array/structured), optionally a timestamp, and other attributes. D2000 supports working with simple values ​​and array values ​​- the first field item is stored in the I/O tag; it is possible to configure the Destination Column for the I/O tag, in which all array items are stored (or the first N, if the structure does not have enough rows). D2000 can read items of template (UDT) and Dataset metrics.
Note: If a metric arrives with the is_null flag set, the I/O tag will have the Invalid value.

Recommended settings for optimizing data flow

• If the MQTT server supports Topic aliases, use MQTT Version = MQTT 5.0, and set Topic Alias Maximum to a value greater than the number of stations on the line (for Payload Type = Sparkplug Edge Node) or greater than the number of topics expected from the MQTT broker (for Payload Type = Text only / JSON). On the MQTT broker side, ensure that its Topic Alias ​​Maximum value is greater than the number of topics that will be sent to the D2000 KOM process, so that numeric aliases can be used instead of text topics.
  Note: the Topic Alias ​​Maximum value that the MQTT broker sends when connecting as a parameter of the CONNACK message is visible in the communication logs:
  09:31:12.086 13-05-2025|D|MQTT> CONNACK Property # 1 Topic Alias Maximum ( 34)
  09:31:12.087 13-05-2025|T|MQTT> Recv:<00><0A>
  09:31:12.088 13-05-2025|D|MQTT> CONNACK Property value= 10
  
  

• Set Subscribe QoS = QoS_0 to not require acknowledgments for PUBLISH messages (unless they are needed for some reason). Additionally, if TLS is also active, this will cause a small packet (e.g., PUBACK for QoS_1) to be encrypted and result in a larger encrypted packet.
  
  

• Use aliases for metrics (for Payload Type = Sparkplug Edge Node): automatically using the Sparkplug Alias Mode=Automatic (HOBJ) parameter or manually using the SAL=alias item in the I/O tag address. If you use manual aliases, use low numbers (aliases 0-127 are encoded as 1 byte).
  
  

• For Payload Type = Sparkplug Edge Node: if there are many changes to the output I/O tags, we recommend setting the Batch Size parameter to a larger value than the default (10) and possibly increasing the Delay parameter in the station's time parameters. This will cause fewer messages to be generated (and each will contain multiple metrics), which reduces the overhead required to transmit one metric. If specific I/O tags change frequently, the message may contain several values ​​for the same I/O tag (with timestamps). The disadvantage is the delay in the values.

Communication line configuration

• Communication line category: TCP/IP-TCP or TCP/IP-TCP Redundant.

• Host: IP address of MQTT server (or redundant addresses separated by a comma or semicolon).
  Note: In Payload Type = Sparkplug Edge Node mode, the Node Control/Next Server metric is supported, which can be used to command the D2000 KOM process to connect to the next MQTT server (if multiple servers are specified).

• Port: the default port number is 1883 or 8883 for the encrypted SSL/TLS variant.

• Line number: unused, set the value to 0.

Note: The default port for the MQTT protocol is 1883 or 8883 for the SSL/TLS version. D2000 KOM contains an implementation of the SSL/TLS protocol variant; it is also possible to configure it by using the stunnel utility http://www.stunnel.org working in a client mode (client = yes). Stunnel running on the same computer as the D2000 KOM should listen to the 1883 local port, and after connecting the D2000 KOM process to the port, it should encrypt the communication using SSL/TLS and send it to the target MQTT server (typically on port 8883).

Note: When using an SSL/TLS connection, you must pay attention to the expiration of the certificates used. The expiration date can be found on Linux with the command

openssl x509 -text -in file.crt | grep "After"

Notes on MQTT broker redundancy

If the TCP/IP-TCP line is configured, one TCP connection is created, which can be directed to one of multiple configured IP addresses.

If the TCP/IP-TCP Redundant line is configured, two TCP connections are created (to 2 MQTT brokers), each of which can be directed to one of multiple configured IP addresses. The D2000 KOM process receives values from both MQTT brokers and sends all values of the output I/O tags to both MQTT brokers. The write is considered successful if the value is successfully sent to at least one MQTT broker (in the case of confirmation, if "Subscribe QoS" > QoS_0, if it is confirmed by at least one of the MQTT brokers). Such a configuration allows you to configure a redundant system in which the D2000 KOM connects to two redundant MQTT brokers and the other side to at least one MQTT broker, while the MQTT brokers do not have to form a "cluster", i.e. synchronize messages with each other (so the simple and free Eclipse Mosquitto can also be used - see configuration example).

Forced disconnection: If all stations on the line are in the simulation mode or the communication is stopped for them, the line will be disconnected (the communication socket will be closed). If the simulation is disabled for at least one station and the communication is not stopped for it (the Parameters tab of the Station type object), the line will be connected again.

In Payload Type = Sparkplug Edge Node mode, if communication on the Edge Node station is stopped, the line will be disconnected (even if Device type substations have communication stopped). If communication on the Device station is stopped and the Edge Node station is communicating, the D2000 KOM will send a DDEATH message and after turning on the Device station, it will send a DBIRTH message.

Communication line parameters

Dialog line configuration - Protocol parameters tab.
They affect some optional protocol parameters. The following protocol line parameters can be entered:

Table 1