How to Test QoS 1 and QoS 2 Delivery Guarantees

To test MQTT delivery guarantees, publish sequence-numbered messages, break the connection while messages are in flight, and count what arrives. QoS 1 passes when no sequence number is missing (duplicates are fine). QoS 2 passes when nothing is missing and nothing is duplicated. Everything else — persistent sessions, acknowledgement packets, broker queue limits — is about making that experiment honest.

This guide shows how to watch the acknowledgement flow, a Python harness that counts gaps and duplicates, the fault injections worth running and how to read the results.

What each level promises

Two details decide most test outcomes. First, QoS is per hop: the publisher's QoS covers publisher → broker, the subscription QoS covers broker → subscriber, and the effective delivery QoS is the lower of the two. A QoS 2 publish to a QoS 1 subscription is a QoS 1 delivery. Second, the guarantees only survive a disconnect if the session survives — otherwise the broker throws the in-flight state away. The QoS levels guide covers the theory; here we test it.

Step 1: watch the acknowledgements

Before automating anything, look at the packets. The Mosquitto clients print every control packet with -d:

# Terminal 1: subscribe at QoS 2 with debug output
mosquitto_sub -h broker.connectmq.com -p 1883 -i testmqtt-qos-sub -t 'testmqtt/qos/demo' -q 2 -d

# Terminal 2: publish one QoS 2 message with debug output
mosquitto_pub -h broker.connectmq.com -p 1883 -i testmqtt-qos-pub -t 'testmqtt/qos/demo' -m 'hello' -q 2 -d

The publisher prints a flow similar to this:

Client testmqtt-qos-pub sending CONNECT
Client testmqtt-qos-pub received CONNACK (0)
Client testmqtt-qos-pub sending PUBLISH (d0, q2, r0, m1, 'testmqtt/qos/demo', ... (5 bytes))
Client testmqtt-qos-pub received PUBREC (Mid: 1)
Client testmqtt-qos-pub sending PUBREL (m1)
Client testmqtt-qos-pub received PUBCOMP (Mid: 1)
Client testmqtt-qos-pub sending DISCONNECT

d0 is the DUP flag, q2 the QoS, m1 the packet identifier. Repeat with -q 1 and you should see a single PUBACK instead of the three-step handshake. On the subscriber side, check the q value in the received PUBLISH — if you subscribed at QoS 1, it says q1 even though the publisher used QoS 2.

For deeper inspection, capture the traffic and filter in Wireshark with mqtt.msgtype >= 3 && mqtt.msgtype <= 7 (PUBLISH to PUBCOMP) or mqtt.dupflag == 1 to see only retransmissions. In MQTT 5, acknowledgements also carry a reason code: a PUBACK with 0x10 (No matching subscribers) is a success, not an error — look codes up in the reason code reference.

Step 2: a gap and duplicate counter

The harness has three parts: a subscriber with a persistent session that logs every sequence number it receives, a publisher that sends a known range and counts acknowledgements, and a short analysis script.

import sys
import paho.mqtt.client as mqtt

QOS = int(sys.argv[1]) if len(sys.argv) > 1 else 1
TOPIC = "testmqtt/qos-test/seq"
log = open(f"received-q{QOS}.log", "a", buffering=1)  # line-buffered: survives kill -9

def on_connect(client, userdata, flags, reason_code, properties):
    print(f"connected: {reason_code}, session present: {flags.session_present}")
    client.subscribe(TOPIC, qos=QOS)

def on_message(client, userdata, msg):
    log.write(f"{msg.payload.decode()} {int(msg.dup)}\n")

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
                     client_id=f"testmqtt-qos-sub-q{QOS}", clean_session=False)
# client.username_pw_set("username", "password")
client.on_connect = on_connect
client.on_message = on_message
client.connect("broker.connectmq.com", 1883, keepalive=10)
client.loop_forever()   # reconnects automatically after a network error

import sys, time
import paho.mqtt.client as mqtt

QOS = int(sys.argv[1]) if len(sys.argv) > 1 else 1
N = int(sys.argv[2]) if len(sys.argv) > 2 else 2000
TOPIC = "testmqtt/qos-test/seq"
acked = 0

def on_publish(client, userdata, mid, reason_code, properties):
    global acked
    acked += 1          # QoS 1: on PUBACK, QoS 2: on PUBCOMP

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
                     client_id=f"testmqtt-qos-pub-q{QOS}", clean_session=False)
client.on_publish = on_publish
client.connect("broker.connectmq.com", 1883, keepalive=10)
client.loop_start()

for seq in range(N):
    client.publish(TOPIC, str(seq), qos=QOS)   # queued by paho while disconnected
    time.sleep(0.005)

deadline = time.time() + 60
while acked < N and time.time() < deadline:
    time.sleep(0.2)
print(f"published {N}, acknowledged {acked}")
client.disconnect()
client.loop_stop()

#!/usr/bin/env bash
LOG=${1:-received-q1.log}; N=${2:-2000}
echo "received lines : $(wc -l < "$LOG")"
echo "unique         : $(cut -d' ' -f1 "$LOG" | sort -u | wc -l)"
echo "duplicates     : $(cut -d' ' -f1 "$LOG" | sort | uniq -d | wc -l)"
echo "with DUP flag  : $(awk '$2 == 1' "$LOG" | wc -l)"
echo "missing        : $(comm -23 <(seq 0 $((N - 1)) | sort) <(cut -d' ' -f1 "$LOG" | sort -u) | wc -l)"

Run python sub.py 1, then python pub.py 1, then ./analyze.sh received-q1.log. Start with a run without faults: it must show 2,000 unique, 0 missing. If the baseline already fails, fix that first (wrong QoS on the subscription, an ACL, a message size limit) before injecting faults.

Step 3: break things mid-flight

Run each scenario while pub.py is sending, then analyze. Use a fresh log file per run.

BROKER_IP=$(dig +short broker.connectmq.com | head -1)
sudo iptables -I OUTPUT -p tcp -d "$BROKER_IP" --dport 1883 -j DROP
sleep 20
sudo iptables -D OUTPUT -p tcp -d "$BROKER_IP" --dport 1883 -j DROP

The blackhole is the more realistic fault: no RST arrives, so the client only notices through keep-alive (about 1.5 × the 10-second keep-alive on the broker side) and then reconnects. On reconnect the subscriber should print session present: True, and the broker should resend unacknowledged messages with DUP=1 followed by the queued backlog. More cut techniques are in testing QoS with a network cut.

The process crash is the interesting QoS 2 case. paho-mqtt keeps its session state in memory, so kill -9 erases the client's half of the QoS 2 handshake. Depending on whether a library hands the message to your code on PUBLISH or on PUBREL, a crash at the wrong moment can produce a duplicate or a missing message. Exactly-once is a property of the protocol between two endpoints that keep their state — if your application must not double-process after a crash, deduplicate on a message ID in your own storage.

Reading the results

• Missing messages at QoS 1/2 after a network cut — the session did not persist: check session present in the log, the client ID (must be identical on reconnect) and the MQTT 5 session expiry.
• Missing messages only after long outages — the broker's per-client queue filled up and it dropped messages by policy (Mosquitto's max_queued_messages, for example). That is a capacity limit, not a QoS bug; see message queue full.
• Duplicates at QoS 2 — check the effective QoS on the subscriber (q2 in mosquitto_sub -d), then look for crashes or a second client sharing the client ID. More causes in duplicate messages.
• Duplicates without the DUP flag at QoS 1 — usually the publisher resent after reconnecting, which creates a new message from the broker's point of view. Normal for at-least-once.
• acknowledged lower than published — the publisher gave up before the backlog drained; raise the deadline or check for a broker-side inflight limit.

A quick manual check

For a fast sanity check without scripts, subscribe at QoS 1 in the online MQTT client, publish from a terminal with mosquitto_pub -q 1 -d, and compare the QoS shown on both sides. Then disconnect the browser client, publish a few more messages, and reconnect with the same client ID and clean session off: the queued messages should arrive.

Watch QoS deliveries in your browser

QoS 1 is proven by zero gaps, QoS 2 by zero gaps and zero duplicates — measured with sequence numbers across real disconnects, not assumed from the spec. Keep the harness in your repository and rerun it whenever you change the broker, the client library or the session settings.

Frequently asked questions