The MQTT protocol (and its different versions) have a lot more features than what is mentioned here. The following is based on my research in developing a simple MQTT client, which can work on an Arduino for the MQTT Light Switch project and interact with the mosquitto broker.

By monitoring the packets, I found that the mosquitto programs seemed to be using the v3.1 protocol instead of the later v3.1.1, so I have based the project around this version. The most obvious difference is in the content of the CONNECT packet. The version of the client that operates on the Arduino is also limited to topic+message lengths of about 125 bytes, because the “remaining length” field in some packets is encoded as a variable byte length; but for values less than 128, this is simply a byte representing the number. The three bytes lost between 128 and 125 are due to non-message data that takes up space in the “remaining length”. Unless you are actually sending long text-based messages, you should find that this is sufficient. It would also be the case that much longer messages would quickly eat into the Arduino’s limited memory.

QoS (Quality of Service):

Three QoS levels are available under MQTT:

• Level 0: receipt not guaranteed (no acknowledgment)
• Level 1: receipt at least once guaranteed
• Level 2: receipt exactly once guaranteed

The MQTT Light Switch project only uses Level 0, as higher levels require more overhead for the Arduino. In any case, if the message is not acted upon immediately, the user can manually resend. Level 2 might be used in something like an instant messaging app, where you’d only want one copy of a message to be received.

RETAIN

A flag can be set when a message is sent to the broker for the message to be “retained”. In this case, any clients connecting or reconnecting will receive the last retained message for a topic. Think of this as “last good known value”, where it is better to have an “old” value than no value at all. At QoS Level 0 (where there is no checking that the message is received), using “retain” means that a newly connecting client will receive a message immediately instead of waiting for another client to publish.

TOPIC

A “topic” can be thought of as a “channel” that a subscribing client can “tune” into. The topic can consist of levels which look like a file hierarchy, such as house/livingroom/lights, with levels separated by the “/” character.

A “publish” message may not contain wildcard characters, while a “subscribe” message can.

The “#” character can stand in for any number of levels, so that subscribing to house/# will receive messages relating to house/livingroom/lights, house/kitchen/lights, house/alarm and even house topics.

The “+” character can stand in for a single level, so that subscribing to house/+/lights will return messages relating to house/livingroom/lights or house/kitchen/lights topics, but not house/livingroom/TV, house/alarm or house topics.

Subscribing to “#” will subscribe to all messages coming through broker.

MESSAGE

The “message” is the content of the data that is sent to a topic. It is effectively a string of UTF-8 characters, but the Arduino in the MQTT Light Switch Project is limited to interpreting these as ASCII. To send a numeric value, it is recommended to send a string representation.

WILL

A client can set up a “will topic” with the broker, which is a topic that is published if the client disconnects unexpectedly (without sending a disconnect message). Think of it in the sense of “last will and testament”. This allows other clients to be informed if another client has gone offline unexpectedly.

The actual execution of the MQTT protocol occurs as a series of TCP-IP packets. The first four bits of the packet identify its function, with many packets only being two bytes long.

A typical exchange (based on what happens in the MQTT Light Switch) will be as follows:

• Client: Connects to Server
• Client: Sends CONNECT packet to request a connection
• Server: Sends CONNACK packet to signal that is accepts the client connection.

A subscribing client will do the following:

• Client: Sends SUBSCRIBE packet
• Server: Records the topics the client wants, and then sends a SUBACK packet
• Server: Sends a PUBLISH packet to propagate the message to subscribed clients.

If the QoS is greater than zero, further acknowledgments may occur.

A publishing client will simply send a PUBLISH packet as necessary:

• Client: Sends PUBLISH packet to the server

If the QoS is greater than zero, further acknowledgments may occur.

To prevent an idle connection being dropped, a client might send a PINGREQ packet:

• Client: Sends PINGREQ packet
• Server: Sends PINGRESP packet

The MQTT Light Switch project is set to do this about every minute to keep the connection alive.

Before closing a connection, a client should send a DISCONNECT packet:

• Client: Sends DISCONNECT packet

If a client disconnects without sending a DISCONNECT packet, its “will topic” will be published to advise other clients that it has disconnected unexpectedly.

The structure of the packets used in the MQTT Light Switch is detailed here, starting with the CONNECT packet. If a server does not receive a CONNECT packet after a client connects to it, it should disconnect that client.

BYTE	VALUE	MEANING
1	0x10	Connect Packet
2	0x0E	14 Bytes Left in Message
3	0x00	Protocol Name Length, MSB
4	0x06	Protocol Name Length, LSB (total 6 bytes)
5	0x4D	"M"
6	0x51	"Q"
7	0x49	"I"
8	0x73	"s"
9	0x64	"d"
10	0x70	"p"
11	0x03	Protocol Version (0x03 Means v3.1)
12	0x02	Flags - Bit 1 Set Means Clean Session
13	0x00	Keep Alive MSB in Seconds
14	0x3C	Keep Alive LSB in Seconds (Total 60 Seconds)
15	0x00	Variable Payload Length MSB
16	0x00	Variable Payload Length LSB (0 Bytes More)

The server responds with a “CONNACK” message, which will look something like:

BYTE	VALUE	MEANING
1	0x20	CONNACK Packet
2	0x02	2 Bytes Left in Message
3	0x00	Session Present Flag (0x00 Means Connection Accepted)
4	0x00	Connected Accepted (Other Valyes Indicate an Error

In practice, the MQTT Light Switch sketch simply checks if the first byte is 0x20, as the server will undoubtedly disconnect if it doesn’t want to connect.

The example SUBSCRIBE packet below corresponds to a subscription to “#” with QoS 0. The MQTT protocol allows multiple topics to be subscribed to in the one packet, but for simplicity, the Arduino code is only capable of one at a time. They can be differentiated by using a different message ID.

BYTE	VALUE	MEANING
1	0x82	SUBSCRIBE Packet with "SUBACK" Expected in Reply
2	0x06	6 Bytes Left in Message - Topic Length +5
3	0x00	Message ID MSB
4	0x01	Message ID LSB (Value = 1)
5	0x00	Topic Length MSB
6	0x01	Topic Length LSB (Value = 1)
7	0x23	#
8	0x00	Requested QoS on this Topic = 0

In the MQTT Light Switch project, there is a function that generates these packets based on topic and ID.

The server should respond with a SUBACK packet:

BYTE	VALUE	MEANING
1	0x90	SUBACK Packet
2	0x03	2 Bytes Left in Message
3	0x00	Message ID MSB
4	0x01	Message ID LSB (Value = 1)
5	0x00	QoS Granted

The MQTT Light Switch simply checks if a 5-byte packet has been received. In practice, the Message ID and QoS should match those requested.

The PUBLISH packet is quite important, and probably the most complicated for the Arduino, as it is should be capable of sending and receiving these. Below is an example for sending the “message” message to the “topic” topic.

BYTE	VALUE	MEANING
1	0x30	Public Packed with Retain = 0
2	0x0E	14 Bytes Left - 2+"Topic"+"Message"
3	0x00	Topic Length MSB
4	0x05	Topic Length LSB (Total 5 Bytes)
5	0x74	"t"
6	0x6F	"o"
7	0x70	"p"
8	0x69	"i"
9	0x63	"c"
10	0x6D	"m"
11	0x65	"e"
12	0x73	"s"
13	0x73	"s"
14	0x61	"a"
15	0x67	"g"
16	0x65	"e"

This is a basic implementation of a QoS level 0 message. Levels 1 and 2 have extra overheads such as message IDs, and would also expect a response in acknowledgment. To send a message with retain=1, the only difference is that the first byte would be 0x31.

To read a publish message that comes from the server, the stream of data is buffered into an array. The first byte is checked to start with 0x3, and the remainder length and topic length are extracted, after which the message length and location are extracted, and the data itself is extracted.

The other two packets that are used are the “PINGREQ” (ping request) and “PINGRESP” (ping response) packets. These are both simply two bytes each, and serve to let the server and client know that the connection is working and should be kept alive. The client sends the “PINGREQ” to the server:

BYTE	VALUE	MEANING
1	0xC0	PINGREQ Packet
2	0x00	0 Bytes Left in Message

And the server should respond with a “PINGRESP”:

BYTE	VALUE	MEANING
1	0xD0	PINGRESP Packet
2	0x00	0 Bytes Left in Message

The function that scans for publish messages is also able to flag PINGRESP packets, as they occur in the stream of data from the server, and shows this in the debugging stream in the serial port.

MQTT is complex, but we can use it in a simple way, suitable for Arduino tasks. For practical implementation, see our MQTT project.

MQTT: Machine Queueing Telemetry Transport - a lightweight messaging protocol using a publish-subscribe model that runs over a TCP-IP network.

BROKER: A device on the network through which all messages are sent. All client devices need only connect to the broker to perform their function.

CLIENT: A device on the network which connects to a broker. A client can choose to subscribe, publish or both.

SUBSCRIBE: A client that wishes to receive messages must first subscribe to a topic, after which the broker will forward messages relating to that topic.

PUBLISH: Messages are propagated around the network by means of “publish” messages. Typically, a client will send a publish message to the broker, after which the broker will send publish messages out to subscribed clients to complete transmission of the message.

A COMPLETE SPECIFICATION OF THE MQTT v3.1 PROTOCOL: http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/MQTT_V3.1_Protocol_Specific.pdf