Tutorials:
Sensors, interfaces and bus systems (SENIN, BUSSY)

MQTT

last updated: 2023-10-23

Quick links

Introduction

Song of this chapter: Rogue Wave > Descended Like Vultures > Publish My Love

MQTT(http://mqtt.org/) is a lightweight publish-subscribe messaging protocol designed to be used with IoT devices (wiki).

MQTT stands for Message Queue Telemetry Transport and one of the main development objectives was to communicate from machine-to-machine (M2M) but also connect M2M to clouds (Big Data).

As HTTP, MQTT runs on top of Transmission Control Protocol / Internet Protocol (TCP/IP) stack. The most recent implementation is MQTT v5.0 (official OASIS standard), based on the earlier v3.1.1 standard. Our software sill uses the v3.1.1 standard, so we stick to v3.1.1.


MQTT OSI

Publishing and subscribing

Like e.g. ROS (Robot Operating System) MQTT uses a publish-subscribe pattern (pub-sub pattern). This is a messaging pattern where messages are not sent directly from transmitter to receiver (point to point), but are distributed by an MQTT server (formerly called MQTT broker).

The MQTT server is the centrepiece of an pub-sub architecture. It can be implemented very simply on an single board computer like the Raspberry Pi or a NAS but naturally also on mainframes or internet server. As the server distributes messages, the server has to be a publisher, but is never a subscriber!

Clients can publish messages (sender), subscribe messages (receiver) or both. A client (also called node) is an intelligent device like a microcontroller, or computer with a TCP/IP stack and software implementing the MQTT protocol.

A publishing client (called publisher) is only loosely coupled to a subscribing client. It does not even know if there are other clients. Same for a receiving client (subscriber) that is interested in a specific types of messages and will subscribe to these messages.

The messages are published under a topic allowing filtering. Topics are hierarchical divided UTF-8 strings. The different topic levels are separated by a slash /.

Let's take a look at the following setup. The photovoltaic power plant is a publisher. The main topic level is "PV". The plant publishes two sub-levels "sunshine" and "data".

"PV/sunshine" is a boolean value (yes/no, could also be 1/0) and is needed by the charging station to know if the electric vehicle should be loaded (only when the sun shines :)). The charging station (EVSE) is a subscriber and subscribes to "PV/sunshine" to get the information from the server.

"PV/data" on the other hand transports the momentary power generated by the plant in kW and that topic could e.g. be subscribed by computers or tablets to generate diagrams of the delivered power over a day.


MQTT publish subscribe

"Just do it" MQTT1:
"Just do it" MQTT2:


MQTT.fx


MQTT_explorer

Topics and message filtering

The topic is are hierarchical divided UTF-8 string. The different topic levels are separated by a slash /, the topic level separator. There is no need to inform the broker over the topics, every syntactically right string is valid. Some examples of topics:

    myhome/firstFloor/Kitchen/temperature
    myhome/firstFloor/Kitchen/humidity
    Luxembourg/Luxembourg/Jofferchersgaass/4/heating/onOff
    6133/49609/btsiot/sc4

The first topic has 4 levels. Topics are case-sensitive, so firstfloor is not the same as firstFloor. Empty spaces are permitted but not very useful as such strings are prone to errors. Each topic must contain at least 1 character, but the forward slash alone is a valid topic.

Topics should not begin with $. The $-symbol topics are reserved for internal statistics of the MQTT broker, even if there is no official standardization. Commonly $SYS/ is used for all the following information as explained in the MQTT GitHub wiki.

It is important to maintain consistency in topic names and we should be aware that we can subscribe to multiple topics by using topic filters. Therefore, it is very important to choose the right topic names for a project.

Wildcards

To filter topics when subscribing, two wildcards can be used, the single-level wildcard: + or the multi-level wildcard: #.

Single-level wildcard +

For all temperatures on the first floor of myhome we could use:

    myhome/firstFloor/+/temperature

For all temperatures in myhome we could:

    myhome/+/+/temperature
Multi-level wildcard #

The the multi-level wildcard must be placed as the last character in the topic and preceded by a forward slash.

For all sensors placed in the Kitchen (first floor) of myhome we could use:

    myhome/firstFloor/Kitchen/#

And for all sensors from the first floor:

    myhome/firstFloor/#

Quality of Service levels

The quality of service is an important feature of MQTT. As we use TCP/IP the connections are already secured at a certain level. But in wireless networks interruptions and disturbances are frequent and MQTT helps here to avoid the loss of information with its quality of service levels. These levels are used when publishing. If a client publishes to the MQTT server, the client will be the sender and the MQTT server the receiver. When the MQTT server publishes messages the client, the server is the sender and the client is the receiver.

QoS 0: fire and forget

This is the normal quality of TCP/IP. There is no acknowledgement from the server and the message is not stored.

MQTT QoS 0

QoS 1: at least once delivery

QoS 1 promises that the message will be delivered at least once to the subscriber. The message gets a message Id. The sender stores the message until it receives an acknowledgement (PUBACK) from the subscriber. The first time the sender sets the duplicate flag to 0 (DUP = 0). QoS 1 tells the destination device (receiver) that a confirmation requirement is necessary. If the message was received correctly the receiver sends a PUBACK (acknowledgement) with the right message Id. However if a certain time is exceeded without confirmation, the message is sent again, this time with the duplicate flag set to 1 (DUP = 1). This can happen more than once to the same destination device (the message is delivered multiple times). This destination device must have the necessary logic to detect duplicates and react likewise, e.g if the receiver is an MQTT server, it sends the message to all clients subscribing the topic and then replies with PUBACK.

MQTT QoS 1

QoS 2: exactly once delivery

With QoS 2 we have a warranty that the message is delivered only once to the destination. For this the message with its unique message Id is stored twice, first from the sender and later by the receiver. QoS level 2 has the highest overhead in the network because two flows between the sender and the receiver are needed. After the first sending (DUP = 0) the sender repeats the sending ((DUP = 1) until it receives a PUBREC that tells, that the message was stored by the receiver. With the second transaction the sender tells the receiver to stop the transmission with a PUBREL (release), to clear the buffer used for the storage and send a PUBCOMP (complete). A published QoS 2 message is successfully delivered after the sender is sure that it has been successfully received once by the destination device.

MQTT QoS 2

More infos can be found here: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/

Local MQTT server on a Raspberry Pi

Using free server in the Internet is simple but also a security risk that is not worth to take if we want to use MQTT e.g. locally in buildings. We will activate a server on a Raspberry Pi (Raspi) single board computer.

We burn an image with RaspiOS on an SD card with the Raspberry Pi Imager. After starting the OS we enable ssh and VNC , change the password and do an upgrade. Finally we give our Raspi a static IP address, so we can reach it. All this was perhaps done in the previous chapter, if the chapters are worked through in order. For more info look here: Using the Raspberry Pi (Raspi).

Install the MQTT server

We will use an an open source (EPL/EDL licensed) message server from Eclipse. The server is called Mosquitto (https://mosquitto.org/) and implements the MQTT protocol versions 5.0, 3.1.1 and 3.1. Mosquitto is lightweight and so can be used on all devices from low power single board computers to full servers.

The installation on a Raspi is done with one command:

    sudo apt install mosquitto

For security reasons the server is not accessible for every one. We have to change some settings. The standard file to do this is the mosquitto.conf file, but we will use our own file in /etc/mosquitto/conf.d. Create the file with: 

    cd /etc/mosquitto/conf.d
    sudo nano mymosqui.conf

Add the following two lines to mymosqui.conf:

    listener 1883
    allow_anonymous true

Restart the service with: 

    sudo service mosquitto restart
"Just do it" MQTT3:

Install an MQTT client on the Raspi and use it

We want to publish messages and subscribe to messages on the same Raspi where the server runs. For this we install a client program on the Raspi:

    sudo apt install mosquitto-clients
Command-line subscription

Start the subscribe client mosquitto_sub with:

    mosquitto_sub -V mqttv311 -t PV/sunshine -d

The Raspi mosquitto server and MQTT.fx use MQTT v3.1.1. So we use the -V option to specify the version. The -t option is needed to specify the topic to which we subscribe, and the -d option stands for debug an gives an verbose output with debug informations.

MQTT sub terminal output

"Just do it" MQTT4:
Command-line publishing

Start the publishing client mosquitto_pub with:

    mosquitto_pub -V mqttv311 -t PV/sunshine -m "yes" -d

The new -m option is needed to specify the message.

MQTT sub terminal output

"Just do it" MQTT5:

MQTT messages and connections

The message format

The MQTT protocol is flexible, trustworthy and efficient. MQTT messages combine a fixed header (all packets), with variable headers (some packets) and payload (some packets).


MQTT message format

Fixed header

The fixed header has 2 byte. The first byte contains the message type and 3 flags (DUP, QoS and RETAIN). The flags are only used in the PUBLISH control packet (for MQTT 3.1.1). The second byte contains the message length (0-127). This is the length of the variable headers and the payload. With none of these the length byte is 0.

If bit 7 of this byte is set, one to three more bytes can be used to store the message length, witch gives a maximum length of 268435455 byte (0xFF, 0xFF, 0xFF, 0x7F). So the overall header can have from 2 to 5 bytes.

message type (name) value description direction
reserved  0 reserved --
CONNECT  1 client request to connect to server client ⟶ server
CONNACK  2 connect acknowledgment client ⟵ server
PUBLISH  3 publish message client ⟷ server
PUBACK  4 publish acknowledge client ⟷ server
PUBREC  5 publish received (assured delivery part 1) client ⟷ server
PUBREL  6 publish released (assured delivery part 2) client ⟷ server
PUBCOMP  7 publish complete (assured delivery part 3) client ⟷ server
SUBSCRIBE  8 client request to subscribe to topic client ⟶ server
SUBACK  9 subscribe acknowledgment client ⟵ server
UNSUBSCRIBE 10 client request to unsubscribe from topic client ⟶ server
UNSUBACK 11 unsubscribe acknowledgment client ⟵ server
PINGREQ 12 client requests PING client ⟶ server
PINGRESP 13 PING response client ⟵ server
DISCONNECT 14 client is disconnecting client ⟶ server
reserved 15 reserved --
Variable headers

With a variable content in the message (topics, client IDs, last will and testament ...) we need a variable header to transport this information. With e.g. 2 topics, we need 2 variable headers. The 2 first bytes contain the length, the following bytes the information. If QoS 1 or QoS 2 are used a packet identifier (consecutive number), is appended.

Payload

The payload length is the message length minus the variable headers length. As the protocol uses a binary transmission, the payload can transport any data not only ASCII.

Delving deeper

With 14 messages we have the all possibilities to login, logout, publish, subscribe and surveillance. Often the client initiates the communication. The server answers with acknowledge messages.

task client server
login/logout CONNECT, DISCONNECT CONNACK
publish PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP
subscribe SUBSCRIBE, UNSUBSCRIBE SUBACK, UNSUBACK
surveillance PINGREQ PINGRESP
Login/logout

The MQTT server is responsible for all connections. It has to authenticate and authorize the MQTT clients. If a client wants to establish a connection with the MQTT server, it must send a CONNECT control packet with a payload that includes all the necessary information to the server. The MQTT server will check the packet. After performing authentication and authorization it sends a CONNACK control packet to the client to signalise the successful connection. If the client sends an invalid CONNECT control packet, the server automatically closes the connection. The server will keep a successful connection open until the client sends a DISCONNECT control packet to the server or the connection is lost.

MQTT connectMQTT disconnect

The CONNECT control packet must include the following fields or flags in the payload:

MQTT ubsubscribe

MQTT sub terminal output

The following fields are optional in the CONNECT control packet:

After a valid CONNECT control packet the server responds with a CONNACK control packet.

The CONNACK packet has a header with the following fields and flags:

Publish

The used packets depend on the Quality of Service (QoS) levels (see above).

MQTT QoS 0MQTT QoS 1MQTT QoS 2

The PUBLISH packet has a header with the following fields and flags:

The payload contains the actual message. As binary transmission is used, the payload can transport any data, and not only strings.

Subscribe

MQTT subscribe

A single SUBSCRIBE packet can request to subscribe to many topics. The SUBSCRIBE packet includes at least one topic filter and QoS (list of topic + QoS). For QoS 1 and 2 we also get a PacketId field.

The SUBSCRIBE packet has the following fields:

If the server gets a valid SUBSCRIBE packet it will respond with a SUBACK packet that confirms the receipt and processing of the SUBSCRIBE packet.

The SUBACK packet has the following fields:

Unsubscribe

MQTT ubsubscribe

The UNSUBSCRIBE packet contains a PacketId in the header and one or more topics in the payload. It is not necessary to include the QoS levels. A single UNSUBSCRIBE packet can request the server to unsubscribe a client from many topics. The server responds with an UNSUBACK packet. It confirms the receipt and processing of the UNSUBSCRIBE packet. The UNSUBACK packet contains the same PacketId in the header that was received with the UNSUBSCRIBE packet.

"Just do it" MQTT6:

MQTT client with Arduino

Let's get practical. We want to create an IoT device, capable of publishing and subscribing. An App on our mobile phone is used to get the information and to give us the possibility to act.

The device should have a light sensor and should be able to switch on an LED lamp and even dim it.

For our device we need a microcontroller or sb-computer that has a TCP/IP stack. And we need an MQTT library for this device. Fortunately there are MQTT client libraries available for the most popular programming languages and platforms. Some of the libraries might not implement all the features of MQTT, so we must look if a library is suitable or not for our needs.

Possible solutions for IoT devices acting as MQTT client (subscriber, publisher or both) are Arduino or ESP boards (with WiFi, Ethernet or both), a Raspberry Pi board, a mobile phone, a laptop, tablet , computer , NAS, server etc..

"Just do it" MQTT7:

The MQTT library

Next we need an MQTT library for Arduino. One of the oldest and stablest libraries is the library of Nick O'Leary called PubSubClient.


MQTT pubsubclient

Publishing in Arduino

To understand how the library works we will look at first at an Arduino sketch that only publishes a message:

    /* mqtt_basic_pub.ino
    * weigu.lu 
    * Basic ESP32 MQTT example to publish a message*/

    #include <WiFi.h>
    #include <PubSubClient.h>

    // WiFi and network settings
    const char *WIFI_SSID = "myssid";       // SSID
    const char *WIFI_PASSWORD = "mypass";   // password

    // MQTT settings
    const char *MQTT_SERVER = "192.168.131.100";
    const char *MQTT_CLIENT_ID = "bussy_mqtt_pub_1";
    const char *MQTT_OUT_TOPIC = "bussy_mqtt_pub";
    const short MQTT_PORT = 1883;  // TLS=8883

    WiFiClient ESP32_Client;
    PubSubClient MQTT_Client(ESP32_Client);

    const unsigned long PUB_DELAY = 3000;    // publish every in ms
    String mqtt_message;
    int message_counter = 0;
    long prev_millis = 0;

    void setup() {
      Serial.begin(115200);
      init_wifi();
      MQTT_Client.setServer(MQTT_SERVER, MQTT_PORT);
    }

    void loop() {
      if (!MQTT_Client.connected()) {
        mqtt_reconnect();
      }
      MQTT_Client.loop();
      if (millis() - prev_millis > PUB_DELAY) {
        prev_millis = millis();
        ++message_counter;
        mqtt_message = "Hello world#" + String(message_counter);
        Serial.println("Publish MQTT message: " + mqtt_message);
        MQTT_Client.publish(MQTT_OUT_TOPIC, mqtt_message.c_str());
      }
    }

    void init_wifi() {
      Serial.println("Connecting to " + String(WIFI_SSID));
      WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
      while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
      }
      randomSeed(micros());
      Serial.print("\nWiFi connected\nIP address: ");
      Serial.println(WiFi.localIP());
    }

    void mqtt_reconnect() {                        // Loop until reconnected
      while (!MQTT_Client.connected()) {
        Serial.print("Attempting MQTT connection...");    
        if (MQTT_Client.connect(MQTT_CLIENT_ID)) { // Attempt to connect
          Serial.println("connected");          
          MQTT_Client.publish(MQTT_OUT_TOPIC, "connected");
        }
        else {
          Serial.println("failed, rc=" + String(MQTT_Client.state()) +
                        " try again in 5 seconds");      
          delay(5000); // Wait 5 seconds before retrying
        }
      }
    }

First we need to include the WiFi library (included in the core package) and the SubPubClient library. Next we initialise some constants as the SSID and the password for our WiFi connection and the IP of our MQTT server. We will use two functions, one to initialise and connect to WiFi (init_wifi()) a and a second to reconnect to the MQTT server if the connection is lost (mqtt_reconnect()). In our main loop we use the function millis() to get a delay of 3 s without blocking the loop.

The reconnect function is a blocking function meaning if a reconnect is not possible the other tasks of the sketch will not be done until a reconnection is possible.

For the message itself we use a String object instead of C-strings (ESP32 has no memory problems). This makes it easier to compose messages. To convert the String object to a C-String we use the method c_str() (the publish method of PubSubClient needs a C-string).

"Just do it" MQTT8:


MQTT.fx

Subscribing in Arduino
    /* mqtt_basic_sub.ino
    * weigu.lu 
    * Basic ESP32 MQTT example to subscribe to a topic*/

    #include <WiFi.h>
    #include <PubSubClient.h>

    // WiFi and network settings
    const char *WIFI_SSID = "myssid";       // SSID
    const char *WIFI_PASSWORD = "mypass";   // password

    // MQTT settings
    const char *MQTT_SERVER = "192.168.131.100";
    const char *MQTT_CLIENT_ID = "bussy_mqtt_sub_1";
    const char *MQTT_OUT_TOPIC = "bussy_mqtt_sub";
    const char *MQTT_IN_TOPIC = "bussy_mqtt_pub";
    const short MQTT_PORT = 1883;  // TLS=8883

    WiFiClient ESP32_Client;
    PubSubClient MQTT_Client(ESP32_Client);

    void setup() {
      pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED
      Serial.begin(115200);
      init_wifi();
      MQTT_Client.setServer(MQTT_SERVER, MQTT_PORT);
      MQTT_Client.setCallback(mqtt_callback);
    }

    void loop() {
      if (!MQTT_Client.connected()) {
        mqtt_reconnect();
      }
      MQTT_Client.loop();  
    }

    void init_wifi() {
      Serial.println("Connecting to " + String(WIFI_SSID));
      WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
      while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
      }
      randomSeed(micros());
      Serial.print("\nWiFi connected\nIP address: ");
      Serial.println(WiFi.localIP());
    }

    void mqtt_reconnect() {                        // Loop until reconnected
      while (!MQTT_Client.connected()) {
        Serial.print("Attempting MQTT connection...");    
        if (MQTT_Client.connect(MQTT_CLIENT_ID)) { // Attempt to connect
          Serial.println("connected");          
          MQTT_Client.publish(MQTT_OUT_TOPIC, "connected");
          MQTT_Client.subscribe(MQTT_IN_TOPIC);         // ... and resubscribe
        }
        else {
          Serial.println("failed, rc=" + String(MQTT_Client.state()) +
                        " try again in 5 seconds");      
          delay(5000); // Wait 5 seconds before retrying
        }
      }
    }

    void mqtt_callback(char* topic, byte* payload, unsigned int length) {  
      Serial.print("Topic: " + String(topic) + "\nMQTT message: ");
      for (int i = 0; i < length; i++) {
        Serial.print((char)payload[i]);
      }
      Serial.println();
      if ((char)payload[0] == '1') {     // Switch LED on if first character is '1'
        digitalWrite(BUILTIN_LED, HIGH); // LED on 
      } else {
        digitalWrite(BUILTIN_LED, LOW);  // LED off
      }
    }

In this example we get have 2 topics, one to publish and one to subscribe. We need the same init_wifi() function, but the mqtt_reconnect() function changes. It must reconnect and subscribe to the in topic. New is the mqtt_callback() function.

"Just do it" MQTT9:
"Just do it" MQTT10:

MQTT security

Security is for the IoT an extremely important topic! Sometimes we we use MQTT to publish values that are neither confidential nor critical for other applications, but often it is important, that only the authorized persons can access a device e.g if we control a drone via MQTT.
Another aspect is that our device perhaps can be used to gain control over the network behind our device. If you search the web you find many of these exploits, like the hack of the Cherokee jeep or the HVAC hack at target.
The novel "Blackout" from author Marc Elsberg published in 2012 shows impressively how hacked smartmeters could be used to trigger a collapse of electrical grids across Europe. The book made clear that we didn't focus enough on security and helped to to draw attention to the problem. It was even noticed by Germany politics.

Security generates always an additional overhead. So it is important to keep a balance to avoid overheads that can make our project unusable. Also some cryptographic algorithms are not suitable for microcontroller with little processing power in our IoT boards. If security is needed we better choose a more powerful hardware for our IoT board. An ESP32 microcontroller for example has a separate hardware accelerator for cryptographic algorithm implementations, that neither an Arduino or ESP8266 can provide.
Many security levels also may require too many maintenance tasks (e.g. generate and distribute a certificate for each device), so the project gets unmaintainable and extremely complex.

For security we need need Authentication to check who (person or device) wants to publish or subscribe (login with name and password) and we need Authorisation to allow the access to all the data or parts of it or to deny the access.

Security on different levels

MQTT has some own security features, but uses also other standardized solutions like SSL/TLS. As MQTT resides in the top layers of the on top of the OSI model we can implement security in different layers:

Authentication with username and password

First let's create a folder named mosquitto in our home folder and change the directory.

    mkdir mosquitto
    cd mosquitto

To create a password file we have the utility mosquitto_passwd. The switch -c is used to create a new password file (overwrites an existing file!) and the switch -b is convenient to run in batch mode and allow passing passwords on the command line. Here an example:
The file is named mosqui_passfile. Two logins with the usernames btsiot1 and btsiot2 are created. The passwords are btsiotpass1 and btsiotpass2.


MQTT create password file

After creating the file we find in in our new folder: /home/pi/mosquitto. We see that the username is cleartext, but the passwords are hashed and not readable!


MQTT password file

Now we need to tell the mosquitto-server to use the password file. This is done in the configuration file.

The default configuration file is in /etc/mosquitto/mosquitto.conf. Here a link to a complete default configuration file: mosquitto.conf. If we open this file we read at the beginning that we should place our own file into the folder conf.d:


MQTT config file

We also see that a log file is already specified in the default file.
Let's use our own configuration file with the name mymosqui.conf in the /etc/mosquitto/conf.d folder:

    # My personal config file for mosquitto

    # Port to listen (1883 is also used with password!)
    listener 1883

    ### Security ###
    # Allow anonymous
    allow_anonymous false
    # Client Prefix:
    #clientid_prefixes btsiot-
    # Path to password file
    password_file /home/pi/mosquitto/mosqui_passfile

    # Logging (don't define "log_dest file" because already defined in default!)
    log_dest stdout
    log_type error
    log_type warning
    log_type notice
    log_type information
    connection_messages true
    log_timestamp true

Now you can use the following commands for two different clients. The -i switch is here optional, but let's us add the client-id if we want to (pay attention, it must be unique):

    mosquitto_sub -V mqttv311 -p 1883 -i mycl1 -u btsiot1 -P btsiotpass1 -t PV/sunshine -d

    mosquitto_pub -V mqttv311 -p 1883 -i mycl2 -u btsiot2 -P btsiotpass2 -t PV/sunshine -m Greetings -d
"Just do it" MQTT11:

Client Id with unique prefix

We can increase security with a unique prefix for the client Id.

"Just do it" MQTT12:

Encryption with TLS-PSK

Even if we use passwords and prefix, the password, client ID and the topics are only encrypted if WiFi is used. In a wired environment it is easy to get these information with a sniffer as wireshark (as seen in the just do it exercise).

The only secure solution is encryption. We have two possibilities. We can use TLS-PSK Transport Layer Security cipher suites with a pre-shared key or TLS/SSL encryption.

The pre-shared keys are symmetric keys (same key on client and server) shared in advance among the clients and server to establish a TLS connection. Pre shared keys are used e.g. in WiFi routers where all the clients need to know the WiFi key in advance. Clearly it is difficult to exchange secret keys with unknown clients over the Internet (e.g. to secure Internet shopping). This is the disadvantage of TLS-PSK, and for such applications TLS/SSL with asymmetric keys (private and public keys) is used.

So what are the advantages of TLS-PSK?
A first advantage of TLS-PSK (depending on the cipher suite) is that there are no public key operations (no key server) needed. So less performant microcontroller can be used. A second advantage is that it is also easier in closed environments (manually configuration in advance) to configure a PSK than to use certificates.

If we send a message of 72 Byte we get 1172 byte with TLS-PSK and 3742 byte with TLS/SSL. So clearly we get less overhead with TLS-PSK.

In MQTT the PSK keys are defined in a PSK file. The name of the file is arbitrary. PSK identities and keys are separated with a colon. The key is in hex and should have more than 20 digits. Let's try this with a file named mosqui_pskfile with the following content:

    btsiot1:0123456789abcdef0123

Now we change the listener port and add two lines to switch on psk-support and provide the path to the psk-file to /etc/mosquitto/conf.d/mymosqui.conf:

    # Port to use for the default listener.
    listener 8883
    # Pre-shared-key based SSL/TLS support (text "btsiot" is arbitrary)
    psk_hint btsiot
    # Path to pre-shared-key key file
    psk_file /home/pi/mosquitto/mosqui_pskfile

The default port for encrypted MQTT communication is 8883. The psk_hint line switches psk support on. The text is needed for authentication by the broker but is arbitrary.

Now we can publish and subscribe with the following lines. Because of our unique prefix the -i switch (ClientId) is not optional.

    mosquitto_sub -V mqttv311 -p 8883 -i btsiot-client1 --psk-identity btsiot1 --psk 0123456789abcdef0123 \
    -u btsiot1 -P btsiotpass1 -t PV/sunshine -d

    mosquitto_pub -V mqttv311 -p 8883 -i btsiot-client2 --psk-identity btsiot1 --psk 0123456789abcdef0123 \
    -u btsiot1 -P btsiotpass1 -t PV/sunshine -m "Hallo_BTS-IoT_:)" -d
"Just do it" MQTT13:
TLS-PSK with Arduino

To use TLS-PSK we include the WiFiClientSecure header file and use an object of type WiFiClientSecure instead of WiFiClient. In setup() we need to add the setPreSharedKey() method to submit the PSK identity and the key. Here is the code to publish a message:

    /* mqtt_basic_pub_psk.ino
    * weigu.lu 
    * Basic ESP32 MQTT example to publish an encrypted message with TLS-PSK0*/

    #include <WiFi.h>
    #include <PubSubClient.h>
    #include <WiFiClientSecure.h>

    const char *WIFI_SSID = "myssid";       // SSID
    const char *WIFI_PASSWORD = "mypass";   // password

    // MQTT settings
    const char *MQTT_SERVER = "192.168.131.100";
    const char *MQTT_CLIENT_ID = "btsiot-client2";
    const char *MQTT_PSK_IDENTITY = "btsiot1";
    const char *MQTT_PSK_KEY = "0123456789abcdef0123"; // hex string without 0x
    const char *MQTT_OUT_TOPIC = "PV/sunshine";
    const short MQTT_PORT = 8883;  // TLS=8883
    const char *MQTT_USERNAME = "btsiot1";
    const char *MQTT_PASSWORD = "btsiotpass1";

    WiFiClientSecure ESP32_SEC_Client;
    PubSubClient MQTT_Client(ESP32_SEC_Client);

    const unsigned long PUB_DELAY = 3000;    // publish every in ms
    String mqtt_message;
    int message_counter = 0;
    long prev_millis = 0;

    void setup() {
      Serial.begin(115200);
      init_wifi();
      MQTT_Client.setServer(MQTT_SERVER, MQTT_PORT);
      ESP32_SEC_Client.setPreSharedKey(MQTT_PSK_IDENTITY, MQTT_PSK_KEY);
    }

    void loop() {
      if (!MQTT_Client.connected()) {
        mqtt_reconnect();
      }
      MQTT_Client.loop();
      if (millis() - prev_millis > PUB_DELAY) {
        prev_millis = millis();
        ++message_counter;
        mqtt_message = "Hello world#" + String(message_counter);
        Serial.println("Publish MQTT message: " + mqtt_message);
        MQTT_Client.publish(MQTT_OUT_TOPIC, mqtt_message.c_str());
      }
    }

    void init_wifi() {
      Serial.println("Connecting to " + String(WIFI_SSID));
      WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
      while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
      }
      randomSeed(micros());
      Serial.print("\nWiFi connected\nIP address: ");
      Serial.println(WiFi.localIP());
    }

    void mqtt_reconnect() {                        // Loop until reconnected
      while (!MQTT_Client.connected()) {
        Serial.print("Attempting MQTT connection...");
        if (MQTT_Client.connect(MQTT_CLIENT_ID,MQTT_USERNAME,MQTT_PASSWORD)) { // Attempt to connect
          Serial.println("connected");
          MQTT_Client.publish(MQTT_OUT_TOPIC, "connected");
        }
        else {
          Serial.println("failed, rc=" + String(MQTT_Client.state()) +
                        " try again in 5 seconds");
          delay(5000); // Wait 5 seconds before retrying
        }
      }
    }
"Just do it" MQTT14:

Encryption with TLS/SSL

Usually, Transport Layer Security (TLS) uses public key certificates. We find many sites on the internet that explain how to use TLS/SSL on the Raspberry Pi with mosquitto or with Arduino. As this topic is already covered in other modules and can be reviewed in the net, we will not go into it further.

Interesting links