What The Tech

How MQTT Works

Tim Blythman

Issue 3, September 2017

The MQTT protocol (and its different versions) have a lot more features than what is mentioned here. The following is based on my research in developing a simple MQTT client, which can work on an Arduino for the MQTT Light Switch project and interact with the mosquitto broker.

By monitoring the packets, I found that the mosquitto programs seemed to be using the v3.1 protocol instead of the later v3.1.1, so I have based the project around this version. The most obvious difference is in the content of the CONNECT packet. The version of the client that operates on the Arduino is also limited to topic+message lengths of about 125 bytes, because the “remaining length” field in some packets is encoded as a variable byte length; but for values less than 128, this is simply a byte representing the number. The three bytes lost between 128 and 125 are due to non-message data that takes up space in the “remaining length”. Unless you are actually sending long text-based messages, you should find that this is sufficient. It would also be the case that much longer messages would quickly eat into the Arduino’s limited memory.

QoS (Quality of Service):

Three QoS levels are available under MQTT:

  • Level 0: receipt not guaranteed (no acknowledgment)
  • Level 1: receipt at least once guaranteed
  • Level 2: receipt exactly once guaranteed

The MQTT Light Switch project only uses Level 0, as higher levels require more overhead for the Arduino. In any case, if the message is not acted upon immediately, the user can manually resend. Level 2 might be used in something like an instant messaging app, where you’d only want one copy of a message to be received.

RETAIN

A flag can be set when a message is sent to the broker for the message to be “retained”. In this case, any clients connecting or reconnecting will receive the last retained message for a topic. Think of this as “last good known value”, where it is better to have an “old” value than no value at all. At QoS Level 0 (where there is no checking that the message is received), using “retain” means that a newly connecting client will receive a message immediately instead of waiting for another client to publish.

TOPIC

A “topic” can be thought of as a “channel” that a subscribing client can “tune” into. The topic can consist of levels which look like a file hierarchy, such as house/livingroom/lights, with levels separated by the “/” character.

A “publish” message may not contain wildcard characters, while a “subscribe” message can.

The “#” character can stand in for any number of levels, so that subscribing to house/# will receive messages relating to house/livingroom/lights, house/kitchen/lights, house/alarm and even house topics.

The “+” character can stand in for a single level, so that subscribing to house/+/lights will return messages relating to house/livingroom/lights or house/kitchen/lights topics, but not house/livingroom/TV, house/alarm or house topics.

Subscribing to “#” will subscribe to all messages coming through broker.

MESSAGE

The “message” is the content of the data that is sent to a topic. It is effectively a string of UTF-8 characters, but the Arduino in the MQTT Light Switch Project is limited to interpreting these as ASCII. To send a numeric value, it is recommended to send a string representation.

WILL

A client can set up a “will topic” with the broker, which is a topic that is published if the client disconnects unexpectedly (without sending a disconnect message). Think of it in the sense of “last will and testament”. This allows other clients to be informed if another client has gone offline unexpectedly.

The actual execution of the MQTT protocol occurs as a series of TCP-IP packets. The first four bits of the packet identify its function, with many packets only being two bytes long.

A typical exchange (based on what happens in the MQTT Light Switch) will be as follows:

  • Client: Connects to Server
  • Client: Sends CONNECT packet to request a connection
  • Server: Sends CONNACK packet to signal that is accepts the client connection.

A subscribing client will do the following:

  • Client: Sends SUBSCRIBE packet
  • Server: Records the topics the client wants, and then sends a SUBACK packet
  • Server: Sends a PUBLISH packet to propagate the message to subscribed clients.

If the QoS is greater than zero, further acknowledgments may occur.

A publishing client will simply send a PUBLISH packet as necessary:

  • Client: Sends PUBLISH packet to the server

If the QoS is greater than zero, further acknowledgments may occur.

To prevent an idle connection being dropped, a client might send a PINGREQ packet:

  • Client: Sends PINGREQ packet
  • Server: Sends PINGRESP packet

The MQTT Light Switch project is set to do this about every minute to keep the connection alive.

Before closing a connection, a client should send a DISCONNECT packet:

  • Client: Sends DISCONNECT packet

If a client disconnects without sending a DISCONNECT packet, its “will topic” will be published to advise other clients that it has disconnected unexpectedly.

The structure of the packets used in the MQTT Light Switch is detailed here, starting with the CONNECT packet. If a server does not receive a CONNECT packet after a client connects to it, it should disconnect that client.

BYTE   VALUE  MEANING
1      0x10   Connect Packet
2      0x0E   14 Bytes Left in Message
3      0x00   Protocol Name Length, MSB
4      0x06   Protocol Name Length, LSB (total 6 bytes)
5      0x4D   "M"
6      0x51   "Q"
7      0x49   "I"
8      0x73   "s"
9      0x64   "d"
10     0x70   "p"
11     0x03   Protocol Version (0x03 Means v3.1)
12     0x02   Flags - Bit 1 Set Means Clean Session
13     0x00   Keep Alive MSB in Seconds
14     0x3C   Keep Alive LSB in Seconds (Total 60 Seconds)
15     0x00   Variable Payload Length MSB
16     0x00   Variable Payload Length LSB (0 Bytes More)

The server responds with a “CONNACK” message, which will look something like:

BYTE   VALUE  MEANING
1      0x20   CONNACK Packet
2      0x02   2 Bytes Left in Message
3      0x00   Session Present Flag (0x00 Means Connection Accepted)
4      0x00   Connected Accepted (Other Valyes Indicate an Error

Want More?

Log in to unlock this article, or sign up now to receive two premium articles free!