Broker Testing

Capture MQTT Traffic with tcpdump

tcpdump recipes for MQTT on servers, containers and gateways: port and packet-type filters, rotating captures, spotting disconnects and analysing pcap files.

Updated · 8 min read

Capture MQTT Traffic with tcpdump

tcpdump is the packet capture tool that is already installed — or one package away — on almost every Linux server, router and gateway. It does not understand MQTT, but it does not need to: capturing the TCP conversation on port 1883 or 8883 is enough to prove who connected, who closed the connection, and whether packets were lost on the way. When you need the MQTT details, you open the same file in Wireshark.

This guide is a set of tcpdump recipes for MQTT: the basic captures, BPF filters that match MQTT packet types, captures in Docker and Kubernetes, long-running ring buffers and how to analyse what you caught.

The basic MQTT capture

Capture MQTT on all interfaces
# Print a line per segment (plain MQTT and TLS)
sudo tcpdump -i any -nn 'tcp port 1883 or tcp port 8883'

# Write to a file for Wireshark instead
sudo tcpdump -i any -nn -w mqtt.pcap 'tcp port 1883 or tcp port 8883'

# Only traffic with one device
sudo tcpdump -i any -nn 'host 192.168.1.50 and tcp port 1883'

The options you will use most:

OptionMeaning
-i any / -i eth0Interface to capture on (tcpdump -D lists them)
-nnDo not resolve host names or port names — faster and unambiguous
-w file.pcapWrite raw packets to a file instead of printing
-r file.pcapRead a capture file
-A / -XPrint payload as ASCII / hex and ASCII
-c 200Stop after 200 packets
-UFlush each packet immediately (needed when writing to a pipe)
-ttttFull date and time on every line

To see something useful straight away, subscribe and publish on a test topic from another terminal and watch the payload in ASCII:

Watch a test message on the wire
sudo tcpdump -i any -nn -A 'tcp port 1883' &
mosquitto_pub -h broker.connectmq.com -p 1883 -t 'testmqtt/tcpdump/hello' -m 'hello from tcpdump'

You will see the topic testmqtt/tcpdump/hello and the payload in clear text inside the PUBLISH segment. On port 8883 you see only encrypted bytes — which is the point of TLS.

Where to capture

The place you capture decides what you can prove. A capture on the client shows what the device really sent and what came back, including retransmissions the application never notices. A capture on the broker host shows what actually arrived, after NAT, load balancers and proxies have done their work. When the two disagree, the problem is in between.

For a stubborn issue, capture on both sides at the same time with -tttt timestamps and compare. If the client sends a PUBLISH that never appears on the broker side, look at the network path; if the broker receives it and the subscriber never gets it, look at the broker's ACLs, queues and logs instead.

Reading tcpdump output

A typical MQTT connection looks like this (addresses shortened):

One short MQTT session
10:15:02.114 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [S], seq 1902...
10:15:02.139 IP 203.0.113.10.1883 > 10.0.0.12.52144: Flags [S.], seq 4471..., ack 1902...
10:15:02.139 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [.], ack 1
10:15:02.140 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [P.], length 35    <- CONNECT
10:15:02.166 IP 203.0.113.10.1883 > 10.0.0.12.52144: Flags [P.], length 4     <- CONNACK
10:15:02.167 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [P.], length 42    <- PUBLISH
10:15:02.168 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [P.], length 2     <- DISCONNECT
10:15:02.168 IP 10.0.0.12.52144 > 203.0.113.10.1883: Flags [F.], ...

The TCP flags carry most of the diagnosis:

  • [S] with no [S.] reply — the SYN never got an answer: firewall, routing or wrong IP. See diagnosing MQTT network issues.
  • [S] answered by [R.] — the host is up but nothing listens on the port (Connection refused).
  • [F.] — an orderly close. The side that sends the first FIN is the side that decided to end the session.
  • [R] — an abrupt reset. If it comes from an address that is neither client nor broker, a middlebox killed the connection.
  • The same [P.] segment repeated after a delay — a retransmission, meaning loss on the path.

Filter by MQTT packet type

The first byte of every MQTT packet encodes its type in the upper four bits. A BPF expression can read that byte at the start of the TCP payload, which lets tcpdump keep only certain packet types:

TypeFirst byte (masked with 0xf0)
CONNECT0x10
CONNACK0x20
PUBLISH0x30
SUBSCRIBE0x80
PINGREQ / PINGRESP0xc0 / 0xd0
DISCONNECT0xe0
Only CONNECT and CONNACK packets (plain MQTT, IPv4)
sudo tcpdump -i any -nn -A \
  'tcp port 1883 and tcp[tcpflags] & tcp-push != 0 and
   ((tcp[((tcp[12:1] & 0xf0) >> 2):1] & 0xf0) = 0x10 or
    (tcp[((tcp[12:1] & 0xf0) >> 2):1] & 0xf0) = 0x20)'

(tcp[12:1] & 0xf0) >> 2 is the length of the TCP header, so the expression reads the first payload byte. The tcp-push condition skips empty ACK segments. Two limits to keep in mind: it only sees the first MQTT packet in each segment (several small packets can share one), and tcp[] offsets work for IPv4 only. It is a filter for narrowing a capture, not a decoder — for exact answers, use Wireshark's mqtt.msgtype filter.

Spot disconnects and reconnects

Two short filters answer the most common production question — "why does this client keep disconnecting?":

Connection lifecycle events
# Every new connection attempt (SYN without ACK)
sudo tcpdump -i any -nn 'tcp port 1883 and tcp[tcpflags] == tcp-syn'

# Every close or reset, with timestamps
sudo tcpdump -i any -nn -tttt 'tcp port 1883 and tcp[tcpflags] & (tcp-fin|tcp-rst) != 0'

Read them together:

  • A FIN from the broker about one and a half keep-alive intervals after the client's last packet means the client stopped sending PINGREQ — see MQTT keep alive.
  • A RST after a long idle period, often at the same interval every time, points to a NAT or firewall timeout — see MQTT NAT timeout.
  • Hundreds of SYNs per second from many devices after an outage is a reconnect storm.
  • A FIN from the broker immediately after a new SYN from another address with the same client ID means a session takeover. Give every test client a distinct ID with the client ID generator.

Long-running captures

Intermittent problems need captures that run for hours without filling the disk. tcpdump can rotate files itself:

Ring buffer and hourly files
# 10 files of ~100 MB each, oldest overwritten
sudo tcpdump -i any -nn -w /var/tmp/mqtt.pcap -C 100 -W 10 'tcp port 1883 or tcp port 8883'

# One file per hour, named by time
sudo tcpdump -i any -nn -G 3600 -w '/var/tmp/mqtt-%Y%m%d-%H%M.pcap' 'tcp port 1883'

Start it in tmux or as a temporary systemd unit, note the time when the problem happens, and pull only the relevant file.

Containers, Kubernetes and remote hosts

A broker container usually has no tcpdump inside. Borrow its network namespace instead:

Docker: capture the broker container's traffic
docker run --rm --net container:mosquitto nicolaka/netshoot \
  tcpdump -i any -nn -U -w - 'tcp port 1883' > broker.pcap
Kubernetes: ephemeral debug container
kubectl debug -it mosquitto-0 --image=nicolaka/netshoot --target=mosquitto -- \
  tcpdump -i any -nn 'tcp port 1883'

The debug container needs permission to capture (root and the NET_RAW capability); restrictive pod security policies may block it. Port mappings and bridge networks have their own pitfalls, covered in MQTT Docker networking issues.

To watch a remote server live in Wireshark, stream the capture over SSH (the remote user needs passwordless sudo for tcpdump):

Live remote capture into Wireshark
ssh admin@gateway.example "sudo tcpdump -i any -nn -U -w - 'tcp port 1883'" | wireshark -k -i -

Microcontrollers such as ESP32 cannot run tcpdump. Capture on the gateway, the Wi-Fi router (OpenWrt has a tcpdump package) or a mirrored switch port, filtered by the device's IP address.

Analysing the capture

tcpdump can do a first pass on a saved file:

Quick analysis of a pcap
# How many connection attempts, and from where?
tcpdump -r mqtt.pcap -nn 'tcp[tcpflags] == tcp-syn' | awk '{print $3}' | cut -d. -f1-4 | sort | uniq -c | sort -rn

# Read plain-text payloads around a timestamp
tcpdump -r mqtt.pcap -nn -tttt -A | less

# Hand over to tshark for MQTT decoding
tshark -r mqtt.pcap -Y mqtt -T fields -e frame.time -e ip.src -e mqtt.msgtype -e mqtt.topic

For anything beyond counting, open the file in Wireshark: it decodes every MQTT packet and can decrypt TLS with a key log. The full workflow is in debugging MQTT with Wireshark.

Confirm a fix with a capture

A capture is also the best regression test for a network fix. After changing keep-alive, a firewall timeout or a load balancer setting, run the disconnect filter for longer than the old failure interval while a client stays connected — for example a subscriber in the online MQTT client on testmqtt/tcpdump/#. No FIN or RST in that window means the fix holds.

Keep a test subscriber connected while you capture

tcpdump turns "the connection drops sometimes" into timestamps, flags and IP addresses. Capture narrowly, let the flags tell you who closed the connection, and switch to Wireshark when you need to read the MQTT packets themselves.

Frequently asked questions

What is the tcpdump command to capture MQTT?

sudo tcpdump -i any -nn -w mqtt.pcap 'tcp port 1883 or tcp port 8883' captures plain and TLS MQTT on all interfaces into a file you can open in Wireshark. Drop -w and add -A to print payloads of plain MQTT directly in the terminal.

Can tcpdump decode MQTT packets?

No. tcpdump shows TCP segments, flags and raw bytes, but has no MQTT decoder. Capture with tcpdump, then open the pcap in Wireshark or run tshark -r file.pcap -Y mqtt to see CONNECT, PUBLISH and other MQTT packets.

How do I run tcpdump for a broker inside Docker?

Attach a tools container to the broker's network namespace, for example docker run --rm --net container:mosquitto nicolaka/netshoot tcpdump -i any -nn port 1883. It sees exactly the traffic of the broker container without installing anything into it.