Testing Session Expiry and Offline Message Queues

A persistent MQTT session lets a device go offline and still receive every QoS 1 message it missed. Whether that works depends on four settings that are easy to get wrong: the client ID, Clean Start, the Session Expiry Interval and the broker's queue limits. The reliable way to test it: subscribe with a persistent session, disconnect, publish, reconnect, then assert Session Present and the exact messages you receive. Then repeat after the session should have expired.

This guide gives you a manual test with mosquitto_sub, a Python script that checks all three behaviors, and a list of the failure patterns you will meet.

What has to be true for a session to survive

The MQTT 5 default for Session Expiry Interval is 0. A client that sets Clean Start false but forgets the interval still loses its session the moment it disconnects. The difference between the two protocol versions is explained in Clean Session vs Clean Start, and the general concept in MQTT persistent sessions.

The broker may also override your value: if it allows less than you asked for, an MQTT 5 broker returns its own Session Expiry Interval in the CONNACK. Always log it.

What the broker stores in a session

• The client's subscriptions (with their QoS and options).
• QoS 1 and QoS 2 messages that arrived while the client was offline.
• In-flight QoS 1 and QoS 2 messages that were not fully acknowledged.
• Optionally QoS 0 messages, if the broker is configured to queue them.

Anything outside this list, such as retained messages, lives independently of the session. A message that has its own message expiry can disappear from the queue before the session does.

Manual test with mosquitto_sub

Pick a unique client ID and topic, then create the session:

mosquitto_sub -h broker.connectmq.com -p 1883 -V mqttv5 \
  -i sess-demo-41c2 -c -x 120 -q 1 \
  -t 'testmqtt/session/41c2/#' -v
# press Ctrl+C after "subscribed"

for i in 1 2 3 4 5; do
  mosquitto_pub -h broker.connectmq.com -V mqttv5 -q 1 \
    -t testmqtt/session/41c2/cmd -m "msg $i"
done

mosquitto_sub -h broker.connectmq.com -p 1883 -V mqttv5 \
  -i sess-demo-41c2 -c -x 120 -q 1 \
  -t 'testmqtt/session/41c2/#' -v -d

With -d you see every packet exchanged. Not every Mosquitto version prints the Session Present flag, so the evidence here is the backlog: msg 1 to msg 5, in order, right after the CONNACK. The Python test below checks the flag explicitly. Now stop the client, publish again, wait longer than 120 seconds and run step 3 once more: the session must be gone and none of the new messages may arrive.

Two things trip people up here. First, the topic in step 3 does not matter for the backlog: the stored subscription delivers it. Second, running step 3 while another client uses the same ID kicks one of them off. That is a session takeover, not a session test.

An automated session test in Python

This paho-mqtt 2.x script runs three tests in one go: the session survives and queues messages in order, the session really expires, and what happens when the queue overflows.

import sys, time, uuid
import paho.mqtt.client as mqtt
from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes

HOST, PORT = "broker.connectmq.com", 1883
RUN = uuid.uuid4().hex[:8]
BURST = int(sys.argv[1]) if len(sys.argv) > 1 else 2000   # for the queue-limit test

def connect(cid, clean_start, session_expiry, inbox=None):
    """Connect with MQTT 5, wait for CONNACK, return (client, connack state)."""
    c = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=cid, protocol=mqtt.MQTTv5)
    state = {}
    c.on_connect = lambda cl, u, flags, rc, props: state.update(
        present=flags.session_present, rc=str(rc),
        expiry=getattr(props, "SessionExpiryInterval", None))   # set only if the broker overrides it
    if inbox is not None:
        c.on_message = lambda cl, u, msg: inbox.append(int(msg.payload))
    props = Properties(PacketTypes.CONNECT)
    props.SessionExpiryInterval = session_expiry
    c.connect(HOST, PORT, clean_start=clean_start, properties=props)
    c.loop_start()
    for _ in range(100):
        if state: break
        time.sleep(0.05)
    return c, state

def close(c, end_session=False):
    props = None
    if end_session:
        props = Properties(PacketTypes.DISCONNECT)
        props.SessionExpiryInterval = 0
    c.disconnect(properties=props)
    c.loop_stop()

def go_offline(cid, topic, session_expiry):
    c, state = connect(cid, True, session_expiry)
    c.subscribe(topic, qos=1)
    time.sleep(1)
    close(c)
    return state

def publish(topic, count):
    pub, _ = connect(f"sess-pub-{RUN}", True, 0)
    info = None
    for i in range(count):
        info = pub.publish(topic, str(i), qos=1)
    info.wait_for_publish(timeout=60)
    close(pub)

def drain(inbox, quiet=2.0):
    """Wait until no new message has arrived for `quiet` seconds."""
    last, since = -1, time.time()
    while time.time() - since < quiet:
        if len(inbox) != last:
            last, since = len(inbox), time.time()
        time.sleep(0.1)

def report(name, ok, detail=""):
    print(f"{'PASS' if ok else 'FAIL'} {name} {detail}")

# Test 1: the session survives a disconnect and queues QoS 1 messages
cid, topic = f"sess-a-{RUN}", f"testmqtt/session/{RUN}/a"
go_offline(cid, topic, 300)
publish(topic, 50)
inbox = []
c, s = connect(cid, False, 300, inbox)
drain(inbox)
close(c, end_session=True)
report("session present after reconnect", s.get("present") is True, s)
report("all 50 queued messages delivered in order", inbox == list(range(50)), f"got {len(inbox)}")
if not s.get("present"):
    raise SystemExit("ABORT: the broker does not keep sessions, the other tests are meaningless")

# Test 2: the session really expires
cid, topic = f"sess-b-{RUN}", f"testmqtt/session/{RUN}/b"
go_offline(cid, topic, 5)
time.sleep(10)
c, s = connect(cid, False, 5)
close(c, end_session=True)
report("session gone after 5 s expiry", s.get("present") is False, s)

# Test 3: what happens when the offline queue overflows
cid, topic = f"sess-c-{RUN}", f"testmqtt/session/{RUN}/c"
go_offline(cid, topic, 300)
publish(topic, BURST)
inbox = []
c, s = connect(cid, False, 300, inbox)
drain(inbox, quiet=3)
close(c, end_session=True)
kept = f"first={inbox[0]} last={inbox[-1]}" if inbox else ""
print(f"INFO queue limit: published {BURST}, delivered {len(inbox)} {kept}")

Run it with python session_test.py or pass a larger burst, for example python session_test.py 5000. On a broker that keeps sessions, a healthy run looks like this:

PASS session present after reconnect {'present': True, 'rc': 'Success', 'expiry': None}
PASS all 50 queued messages delivered in order got 50
PASS session gone after 5 s expiry {'present': False, 'rc': 'Success', 'expiry': None}
INFO queue limit: published 2000, delivered 1000 first=0 last=999

The last line is information, not pass/fail. It tells you the effective queue limit and the drop policy: first=0 last=999 means the broker kept the oldest messages and dropped the newest; first=1000 last=1999 would mean the opposite. If expiry shows a number, the broker overrode the interval you asked for.

The script ends each session with Session Expiry Interval = 0 in the DISCONNECT, so test runs do not leave orphaned sessions behind. You can only lower the interval to 0 this way if it was non-zero on CONNECT. Raising it from 0 in DISCONNECT is a protocol error.

Testing queue limits on your own broker

Every broker caps how much it stores per offline client, because a thousand dead devices with unlimited queues would exhaust its memory. In Mosquitto the relevant settings are:

persistence true
persistence_location /mosquitto/data/

# remove persistent sessions of clients offline for more than 2 days
persistent_client_expiration 2d

# per-client offline queue (0 bytes = no byte limit)
max_queued_messages 1000
max_queued_bytes 0

# QoS 0 messages are not queued for offline clients unless enabled
queue_qos0_messages false

Test the limit you configured, not the one you assume: publish a burst of max_queued_messages + 500, reconnect and compare. Then decide what your devices need. A sensor that only cares about the latest command should use message expiry or retained state, not a huge queue. When clients see errors about full queues, MQTT message queue full covers the fixes. The Mosquitto config generator can produce these settings for you.

Also test a broker restart while the client is offline. Without persistence enabled, Mosquitto keeps sessions only in memory, and a restart loses them all.

Failure patterns and what they mean

For the broader debugging walk-through, see why an MQTT session is not persisting.

Testing in the browser

For a quick visual check, open the online MQTT client, connect with a fixed client ID, Clean Start off and a session expiry, subscribe at QoS 1, then disconnect. Publish a few messages from a terminal, reconnect in the browser and watch the backlog arrive.

Try a persistent session in the browser

A session test takes a minute to run and saves hours of debugging "my device missed the command" reports. Keep the script in your repository, run it against every broker you deploy, and treat the queue-limit line as part of your capacity plan.

Frequently asked questions