Architecture
MASQUE is not one protocol but a set of IETF specifications that put proxy tunnels on top of HTTP/3. A classic HTTP proxy gives you one TCP tunnel per connection; here one QUIC connection multiplexes arbitrarily many TCP and UDP tunnels, and looks like ordinary HTTP/3 traffic from the outside.
CONNECT-UDP (RFC 9298) UDP proxying semantics
classic CONNECT (RFC 9114 §4.4) TCP proxying semantics
HTTP Datagrams + Capsules (RFC 9297)
Extended CONNECT (RFC 9220)
HTTP/3 (RFC 9114) + QPACK (RFC 9204)
QUIC (RFC 9000) + DATAGRAM frames (RFC 9221)
TLS 1.3 (RFC 8446 / RFC 9001) over UDP
RFC 9298 stands on 9297 and 9220; 9297 stands on 9221. The TCP path needs none of them — only RFC 9114 itself.
The path of a request
quic.rs owns the quinn endpoint. Transport parameters come from [limits], so
the stream cap, idle timeout, keep-alive, MTU settings, congestion controller and
initial RTT are configuration rather than constants; they apply to connections
accepted from then on, including across a SIGHUP reload. The stream cap is the
one of them that does not reach the handshake as itself; what an unauthenticated
connection is worth is further down this section.
The UDP socket underneath is bound here rather than by quinn::Endpoint::server,
which is a bare std::net::UdpSocket::bind and never sets SO_RCVBUF/SO_SNDBUF
— a server that does not ask keeps net.core.rmem_default whatever rmem_max
says. limits.socket_recv_buffer / socket_send_buffer are requested at that
moment and the granted sizes read back, so a capped request becomes a startup
warning rather than silent packet loss. Being a property of the socket rather
than of a connection, these two are the only [limits] keys a reload cannot
change.
The SNI gate
gate.rs sits between that socket and quinn. It is an AsyncUdpSocket wrapper,
so it sees every datagram before the QUIC layer does, and it is off unless
[security] expected_sni names at least one host — an empty list, the shipped
default, passes everything through untouched.
It exists because quinn’s endpoint is conservative but not silent. A long-header
packet naming a version it does not speak draws a Version Negotiation packet; an
Initial whose Destination Connection ID is shorter than the eight bytes RFC 9000
§7.2 requires draws a CONNECTION_CLOSE; a short-header packet for no live
connection can draw a Stateless Reset. All three are correct answers to a QUIC
peer, and all three are answers to a port scan. With the gate open, a datagram
that would draw one is discarded first.
The judgement is made on the first packet in the datagram, which is enough for
the whole of it: RFC 9000 §12.2 both lets a receiver “route based on the
information in the first packet contained in a UDP datagram” and forbids a sender
from coalescing packets with different connection IDs into one. A short header passes — the gate
holds no connection state to judge it with. A long header naming a version other
than 1 is refused. A long header that is not an Initial passes, because quinn
drops those for an unknown connection without a word. An Initial in a datagram
below 1200 bytes passes for the same reason: RFC 9000 §14.1 has the server
discard it, and quinn does, silently. What is left is an Initial that could start
a connection. Its Destination Connection ID is judged first, on the header alone
and before any decryption: RFC 9000 §7.2 gives a client’s first Initial a
Destination Connection ID of at least eight bytes, every later Initial of an
admitted handshake is addressed by the eight bytes this endpoint chose, and so is
the one a Retry supplies, so nothing legitimate is shorter. quinn applies the
same floor ahead of its own decryption, in early_validate_first_packet. A
shorter one is refused here, which is the CONNECTION_CLOSE above never sent.
The packet is then opened: the keys for it are derived from the Destination
Connection ID in the packet itself (RFC 9001 §5.2), so no connection state is
needed to read it. Its CRYPTO frames are assembled from offset zero,
the ClientHello is parsed, and the server_name extension (RFC 6066 §3) is
compared with the configured list — ASCII case-insensitively, with a trailing
root dot ignored, name for name and no wildcards.
A refused datagram is not removed from the receive batch, because one buffer can hold a whole GRO run described by a single stride. It is left in place with its Destination Connection ID length byte overwritten by a value larger than the 20 bytes RFC 9000 §17.2 allows, which quinn’s own packet decoder rejects before it reads the version, the type or anything else — and rejects by returning rather than by answering.
Two deviations from a SHOULD are deliberate and are the point of the feature.
RFC 9000 §5.2.2 says a server “SHOULD send a Version Negotiation packet” for a
large enough packet naming an unsupported version; with the gate open it sends
nothing. RFC 6066 §3 says a server that does not recognise the name “SHOULD take
one of two actions: either abort the handshake by sending a fatal-level
unrecognized_name(112) alert or continue the handshake”; the socket-level check
does neither, and the second check below aborts with a different alert.
That second check is in tls.rs. The gate reads only what arrives in the first
Initial packet, and a ClientHello large enough to be split across several of them
is passed through on purpose — refusing a first flight whose extensions have not
arrived would make a large ClientHello unreachable. Such a handshake is stopped
by a ResolvesServerCert that declines to present a certificate for a name that
is not on the list, which rustls turns into a fatal access_denied alert. It is
not a certificate selector: there is still one certificate and one key, and
naming several hosts means answering to all of them with the same certificate.
Stateless resets are deliberately left uncovered — see D106 and the configuration reference for what that leaves visible and why filtering on the address instead would break connection migration. So is the acknowledgement quinn returns for an Initial the gate passes that carries an ack-eliciting frame and no name — a PING on its own, or a ClientHello the gate could not finish reading — because refusing those means refusing a large first flight, which this design will not do. None of this is traffic obfuscation, which stays a non-goal: the gate hides that a service is here from somebody who does not know the name to ask for, and says nothing about what a connection looks like once one is open.
Each accepted connection is handed to h3api::Connection::handshake, which must
advertise both SETTINGS_ENABLE_CONNECT_PROTOCOL (0x08) and
SETTINGS_H3_DATAGRAM (0x33). Surge checks for both and disconnects if either
is missing; this is the first thing to suspect when a client drops immediately
after a successful TLS handshake. The frame also carries
SETTINGS_MAX_FIELD_SECTION_SIZE (MAX_FIELD_SECTION_SIZE = 64 KiB), both
QPACK settings as zero, and one reserved “grease” identifier of RFC 9114
§7.2.4.1’s 0x1f * N + 0x21 form, which that section says endpoints SHOULD send
so peers keep exercising the rule that unknown identifiers are ignored.
conn.rs runs the accept loop and dispatches each request stream on the
:protocol pseudo-header, after authenticating it:
- absent, method CONNECT →
tunnel/tcp.rs connect-udp→tunnel/udp.rs- anything else → 501
Authentication deliberately runs before routing, so an unauthenticated client
gets 407 rather than learning from a 501 which :protocol values this server
implements — but after the connection-specific-field check described below, so
a peer that sends one of those fields gets 400 whether or not it has
credentials.
That 501 is a real one for every :protocol value, not only the ones this
server has heard of. The token is carried through as the bytes that arrived, so
connect-ip, webtransport and anything else are answered with the status
RFC 9220 asks for and logged under the name the client actually sent, rather
than being refused as malformed before anything can look at them.
The routing above and the per-connection context both live in tunnel/mod.rs,
beside the target resolution the two tunnel kinds share. Two concerns that are
neither are files of their own: tunnel/status.rs holds every refusal and the
RFC 9209 Proxy-Status vocabulary refusals are written in, and
tunnel/quota.rs the per-connection tunnel budget and the guards that spend it.
Every wait that stands between a packet arriving and a tunnel opening is
bounded, whatever the connection’s authentication state, and two limits share
the job. All but two of those waits are bounded by limits.max_idle_timeout:
the QUIC/TLS handshake in quic.rs, the HTTP/3 handshake and the read of each
peer unidirectional stream’s type in h3/connection.rs, the read of a request
stream’s HEADERS in Resolver::resolve, and every response this server writes
before a tunnel starts relaying — 400, 403, 407, 431, 501 and the 5xx family,
the 200 that closes on the spot, and the 200 that opens a tunnel — in
Stream::respond_within. The two exceptions are resolving the target’s name and
connecting to it, which draw on limits.connect_timeout instead: the lookup
gets that budget once, the connect gets it once for the whole list of addresses
rather than per address, and 0 hands both waits back to the operating system.
The idle-timeout bound is needed because the keep-alive PINGs this server sends
are answered by the peer’s QUIC stack with no application ever involved, so the
transport’s own idle timer never fires on a peer that is present but says
nothing; the response writes need it because a peer that grants no flow-control
window never takes even the fifty-odd bytes of a 407. A tunnel’s own 200 is
bounded like the rest: the pumps that would otherwise end the wait do not exist
until that write returns, and a lapsed one resets the stream and drops the
target socket the request had already opened, with a reset rather than a FIN.
One further bound applies until a request on a connection has passed the
credentials check: twice limits.max_idle_timeout for one request to
authenticate. It is armed once at the handshake and never re-armed by a new
stream, so what it bounds is the connection rather than a pause in it; twice, so
that the transport’s own idle timeout stays the first thing to fire on a peer
that has simply gone away. The clock is read on each pass as well as awaited,
because awaiting it alone is not enough: a timeout only reports a lapsed
deadline on a poll where the work underneath it was not already finished, so a
peer with another request stream always queued — one opened and finished empty
costs it nothing and the server nothing but a stream error — was never measured
against the deadline at all. Without it a peer that finishes the QUIC handshake
and then says nothing holds a max_connections slot for as long as it keeps its
socket open. It is lifted for the life of the connection the moment one request
authenticates, so a client reusing an idle connection between requests is
untouched. A lapsed connection bound closes the connection with H3_NO_ERROR,
which is not an error and is logged as the idle ending it is; a lapsed HEADERS
bound resets that one stream with H3_REQUEST_INCOMPLETE and a lapsed response
write resets it with H3_REQUEST_CANCELLED, leaving everything else on the
connection running.
That bound is on time. The second one on an unauthenticated connection is on
size, and it is a transport parameter: the handshake advertises
INITIAL_BIDI_STREAMS = 16 concurrent bidirectional streams rather than the
configured limits.max_streams_bidi, and the first request to pass the
credentials check raises the connection to the configured value in one step.
Before it, a peer that has proved nothing holds at most 16 request streams — 16
parked request tasks and 16 refusals to write, where 1024 of each were free —
and its whole HEADERS buffering is bounded twice over by the same number, since
16 field sections at the MAX_FIELD_SECTION_SIZE = 64 KiB per-frame cap are
exactly the HEADERS_BUFFER_BUDGET = 1 MiB budget below. Running into the
allowance is backpressure rather than an error: the peer’s own stack holds the
stream, and closing streams returns credit before authentication as after it. A
client that fires more than 16 CONNECTs at once therefore waits for the raise,
which happens before its first request has resolved a name or opened a socket,
so what it waits is the round trip it was already waiting for that first answer.
An operator who configures fewer than 16 gets what they configured throughout:
the clamp only ever lowers.
That bound holds one connection; it does not bound how many of the
limits.max_connections slots unauthenticated peers hold between them, and a
peer that completes a handshake about once a second and never sends a credential
would fill every one of them legitimately, each slot bounded, all of them
replaced. So at the cap a new connection does not simply lose: the endpoint
takes the slot of the oldest connection that has never authenticated and admits
the newcomer, and only refuses when every live connection has authenticated. The
evicted connection is closed with H3_NO_ERROR — nothing about it failed — and
logged like the other idle endings, with reason=evicted; one whose QUIC
handshake had not finished has already been accepted at the transport layer, so
what its peer receives is a CONNECTION_CLOSE carrying APPLICATION_ERROR and
an empty reason rather than the CONNECTION_REFUSED a refusal sends. A
connection that has authenticated is never a candidate, and below the cap
nothing of this runs. There is no sub-quota for unauthenticated connections
because it would not help: a legitimate client is unauthenticated at accept time
too, so it would be squeezed by exactly the pool it is trying to join.
Evicting on the word of an unverified source address would be worse than the
problem: a spoofed Initial costs an attacker one datagram and no return path, so
a flood of them would empty the roster and start a TLS server flight per packet.
At the cap, a newcomer whose address QUIC has not yet validated is therefore
answered with a Retry (RFC 9000 §8.1) — it takes no slot and no crypto — and
only the newcomer that comes back with the token may evict anybody. The extra
round trip is charged to an address the server holds no token from: quinn’s
bloom feature is on, so the server sends NEW_TOKEN frames on every connection
it validates (RFC 9000 §8.1.3) and a client that kept one comes back already
validated and evicts without a Retry — first contact, a token past its two-week
lifetime, or a token sealed before the last restart or SIGHUP are what pay.
Below the cap no Retry is sent at all and the handshake is what it always was.
Target address selection
Both tunnel kinds resolve their target through one function in tunnel/mod.rs,
which is also where the resolved list is ordered by
limits.ip_family_preference — before the destination policy filters it and
before either tunnel dials anything, so the two cannot disagree about which
family goes first. The default is IPv4-first, deliberately unlike getaddrinfo, which
orders by RFC 6724 and so puts global IPv6 ahead of IPv4 on any host with an
IPv6 route. That ordering is an operator policy on a proxy rather than a
resolver detail: the TCP path walks the list in order, so the non-preferred
family costs a full connect attempt, and the CONNECT-UDP path has no failover at
all — connecting a UDP socket only asks the kernel for a route, so the first
address with one wins outright. The reorder is a stable partition, leaving
RFC 6724’s ordering within each family intact; system opts out of it entirely.
The name lookup budget
Name resolution is the one place in this server where a request reaches a
resource it cannot give back on demand. tokio::net::lookup_host runs
getaddrinfo on the runtime’s blocking thread pool, and that call cannot be
cancelled: when limits.connect_timeout expires the client is answered, but the
thread stays in the resolver until the stub gives up on its own — five seconds
per attempt on a stock resolv.conf, tens of seconds when the nameserver
answers nothing at all. The pool is process-wide, so without a bound one
client’s black-holed names park threads that every other connection’s lookups
need, including names that would have resolved from /etc/hosts without a query.
So the pool is rationed in two tiers, in net.rs:
- One reserved lookup slot per connection, which nothing else can take. No number of hostile connections can starve a connection outright; at worst its lookups run one at a time — a lookup queued for the shared allowance keeps its claim on the reserved slot too, and takes it the moment it frees.
- A server-wide allowance on top of that, capped per connection, so ordinary use resolves a page’s worth of tunnels in parallel while no single connection can hold more than its share.
The permit is dropped by the blocking task rather than by the request that asked
for it — a permit released when the client is answered would say a thread is
free while it is still inside the resolver, which is exactly how a client could
park the pool one connect_timeout at a time. An IP literal never reaches the
resolver and never takes a permit.
The blocking pool is sized from limits.max_connections at startup for the same
reason: it has a thread for every slot the budget can hand out, plus headroom.
Threads are created on demand and reaped when idle, so this is a ceiling rather
than an allocation. Because it is fixed when the process starts, a SIGHUP that
raises max_connections does not resize it.
TCP tunnels
tunnel/tcp.rs resolves :authority explicitly — not implicitly through
TcpStream::connect((host, port)), because the destination policy has to see the
resolved address — answers 200, and then runs two independent pumps with full
RFC 9114 §4.4 half-close semantics:
| Event | Behaviour |
|---|---|
| client finishes the stream | shutdown(Write) on the target socket, not a full close |
| target EOF | finish the send side of the response stream |
| target RST or error | reset the stream with H3_CONNECT_ERROR |
| client resets the stream, or stops reading it | close the TCP connection with a reset, and cancel the direction the client left alone with H3_REQUEST_CANCELLED |
Collapsing this into “one side closes, everything closes” breaks every protocol
that depends on half-close. The abnormal paths coordinate through a sticky
watch teardown channel so that neither pump can strand the other.
While both directions are live, each pump is the other’s watchdog and nothing in
a tunnel is on a timer: whatever one pump notices — a reset stream, a target
that fails — raises a teardown, and that teardown is what ends the other’s wait.
The first two rows above are the endings that take the watchdog away, because
they end a direction without a teardown. So from the moment one direction has
ended cleanly, each write in the surviving direction is bounded by [limits] udp_session_timeout — the same knob a CONNECT-UDP session’s idle bound comes
from — and a write that does not complete within one of those cuts the tunnel:
the request stream is reset (H3_REQUEST_CANCELLED when it is the client that
stopped reading after its own FIN, H3_CONNECT_ERROR when it is the target that
stopped reading after its own EOF) and the target socket is closed abortively —
though on the client-FIN path the proxy’s clean FIN has already reached the
target, and a kernel that does not reset from FIN_WAIT_2 (Linux) closes silently
behind it, so only the resources are reclaimed there. Every write gets its own
budget, so a client that keeps reading may take hours to drain a download after
its FIN — but progress alone does not save a write that never finishes, which
puts a floor of roughly one RELAY_BLOCK_SIZE = 64 KiB relay chunk per budget
under how slowly a half-closed download may be drained (about one 1.4 KB piece
per budget in the other direction). A tunnel whose two directions are both still
open is never on a timer at all, however long a write parks.
The bound covers the parked write half of the problem and only that. A
half-closed tunnel whose surviving direction is parked in a read — no bytes
in either direction — is deliberately not put on a timer, because a target that
has taken the client’s FIN may legitimately take minutes to answer and cutting it
would break the half-closes this proxy exists to carry; what limits those is
capacity rather than time, max_connections × max_targets_per_conn sockets.
Without the write bound, a half-closed tunnel whose surviving peer stopped taking
bytes held the target socket, its file descriptor and the tunnel slot until the
QUIC connection ended — which this server’s own keep-alives can postpone
indefinitely.
Not on a timer is not unwatched. Such a tunnel still ends the moment the client
says it wants no more of it: the response direction watches for the client’s
STOP_SENDING — the other half of what dropping a request stream sends — even
while there is nothing to write, so a client that half-closed and then walked
away releases the target socket and its tunnel slot at once rather than at the
end of the QUIC connection. Only a client that is still waiting for an answer
keeps a quiet half-closed tunnel alive.
The last row cancels both directions because RFC 9114 §4.4 asks for it: “If the stream is reset or reading is aborted by the client, a proxy SHOULD perform the same operation on the other direction in order to ensure that both directions of the stream are cancelled.” Without it a client that reset only its sending side had the response direction finished with a clean FIN, so a truncated response read as a complete one, and a client that sent STOP_SENDING had its own sending side stopped with code 0 rather than with an HTTP/3 code.
UDP tunnels
tunnel/udp.rs parses the RFC 9298 well-known path template. Surge has no URI
template parameter, so it uses the default template the RFC defines for exactly
that case:
https://$PROXY_HOST:$PROXY_PORT/.well-known/masque/udp/{target_host}/{target_port}/
Parsing is deliberately lenient: percent-decoding, an optional trailing slash, a
port in 1..=65535 and an empty query. IPv6 literals arrive without brackets,
with the colons escaped (2001:db8::42 → 2001%3Adb8%3A%3A42) per RFC 9298
§3; the bracketed form is also accepted.
Two ordering rules that are easy to get wrong:
- Answer 2xx immediately — do not wait for the target to send anything. UDP is connectionless and the proxy cannot know whether the target is reachable.
- But resolve DNS first. When
target_hostis a name, resolution MUST complete before the response; a failure is refused with 502 andProxy-Status: volto; error=dns_error(RFC 9209). “Immediate” means “without waiting for the target”, not “without resolving”.
The target socket is connected, so packets from anywhere else are dropped by the
kernel. Sessions have their own idle timeout (default
DEFAULT_UDP_SESSION_TIMEOUT = 180 s; RFC 9298 §3.1 asks for at least 120),
where idle means no packet crossed the proxy: the clock is re-armed by a payload
reaching the target or the target answering, never by bytes alone, so a peer
dripping fragments of a capsule that never completes is reclaimed on schedule.
As on the connection bound above, that clock is read on each pass and not only
awaited: a peer with another payload always queued — the ones the amplification
cap drops re-arm nothing, and cost it only bandwidth — kept the session’s wait
finishing on its own and the deadline was never measured against it. Closing the
socket must also close the request stream.
Datagram routing
Each connection routes inbound datagrams to per-session channels from its own
background task, in src/h3/connection.rs — the same task that reads the peer’s
unidirectional streams, since both belong to the connection and both end with
it. A peer may hold MAX_PEER_UNI_STREAMS = 16 of those open at once (a
transport parameter rather than a configuration key) where HTTP/3 needs three,
and each one’s type has to arrive within an idle timeout. Routing is per
request stream, which is why it lives in the HTTP/3 layer: a session claims
its Quarter Stream ID by asking its stream for a DatagramReceiver and holds
the claim exactly as long as it holds that receiver, so a session that ends —
however it ends — takes its routing entry with it. Sending is the other half and
stays outside: a UDP session writes its datagrams straight onto the
quinn::Connection.
Two fields decide where a packet goes, and getting either subtly wrong is silent:
- Quarter Stream ID = stream ID ÷ 4 (RFC 9297 §2.1), not the stream ID. Client-initiated bidirectional stream IDs are always multiples of four, so the low two bits carry no information and dividing saves encoding space. The old draft term “Flow Identifier” is gone, and there is no separate mapping table. A value no session owns is dropped, since a session can close with packets still in flight — but a value that cannot be a stream ID at all (above 2⁶⁰-1, or a datagram too short to parse one) closes the whole QUIC connection with H3_DATAGRAM_ERROR (0x33), which RFC 9297 §2.1 states as a MUST.
- Context ID, a varint at the head of the payload (RFC 9298 §5).
0means a raw UDP payload; anything else is an extension. An unknown Context ID is dropped silently — never a connection error — and so is a truncated one, which no requirement covers.
Every such drop — an unknown Context ID, a truncated one, an unclaimed Quarter
Stream ID, or a session’s inbound queue already full — is counted, and the
total is reported as dropped_datagrams= on the connection’s closing log line,
since the drops themselves are required to be silent.
Get either wrong and the symptom is “the handshake succeeds, the tunnel is
established, and not one packet gets through”, with no error anywhere. The
encoder and decoder are hand-rolled in src/datagram.rs — about thirty lines,
which is why no dependency is carried for them.
Routing uses a bounded channel and try_send: a session that is not draining
its queue loses packets, which is correct UDP behaviour, instead of blocking the
routing task and starving every other session on the connection. A CONNECT (TCP)
tunnel never claims a receiver, so it never appears in the table at all.
The capsule stream
The request stream of a CONNECT-UDP tunnel is not an empty stream. Its body is a
sequence of capsules (RFC 9297 §3), each Type varint + Length varint + Value,
handled by src/capsule.rs as an incremental TLV decoder:
- Requests and responses carry
Capsule-Protocol: ?1; a response using capsules must not carryContent-Length,Content-TypeorTransfer-Encoding. - Unknown capsule types are skipped by reading the length and discarding the value edge-to-edge, with no accumulation — a peer declaring a 2⁶²-byte capsule costs no memory. A truncated capsule is a malformed message.
- The DATAGRAM capsule (type 0x00) is the reliable fallback channel for when QUIC datagrams are unavailable. It is a fallback, not a downgrade: a target packet too large for a QUIC datagram is dropped rather than re-routed through the capsule stream, which would break end-to-end unreliability and defeat the inner path-MTU discovery.
Treating the stream bytes as opaque is a worse trap than the Quarter Stream ID: the decoder desynchronizes on the first capsule that arrives.
RFC 9297 §2.1.1 also forbids sending QUIC datagrams before the peer has
advertised SETTINGS_H3_DATAGRAM. The flag is one shared atomic rather than a
value each session snapshots: the task that reads the peer’s control stream
writes it and every session reads it once per packet, so a session opened before
the SETTINGS arrived moves onto QUIC datagrams the moment they do. The module
comment on src/h3/connection.rs records what the sampled version cost.
The HTTP/3 layer
HTTP/3 is implemented in src/h3 rather than taken from a crate. What it covers
is what a CONNECT proxy needs, and it is stated in full:
- Framing (RFC 9114 §7) — an incremental decoder that hands DATA payload on
in the chunks quinn delivered it in, buffers every other frame because none of
them can be acted on piecewise, and skips an unknown frame type by its declared
length without allocating for it. A frame’s type is judged before its
length: the decoder knows which of the three kinds of stream it is reading —
the peer’s control stream, a request stream, or a request stream whose CONNECT
has been answered — and a frame type that may not appear there is a connection
error at any size. The number of reserved or unknown frames a request stream
may make the decoder skip before its request arrives is bounded too (RFC 9114
§10.5): grease and padding are legitimate (§7.2.8), but a flood of them — zero-
length frames included, which no size bound would catch — is refused for
H3_EXCESSIVE_LOAD. The limit is generous, per stream, and answered with a bare reset rather than a 431, since no field section was ever too large. - QPACK (RFC 9204) against the static table, including Huffman decoding (RFC 7541 Appendix B).
- The control stream — SETTINGS, GOAWAY, and the rules about what may appear
there, read by a task of its own for as long as the connection lasts. RFC 9114
§6.2.1 makes a control stream that is closed at any point a connection error,
and that holds in both directions: a peer that finishes its own, and a peer
that stops this endpoint’s with STOP_SENDING, are each answered with a
CONNECTION_CLOSE carrying
H3_CLOSED_CRITICAL_STREAM. - Request validation — RFC 9114 §4.1.2, §4.3 and §4.4 plus RFC 8441 §4: the point at which a field section becomes a request or is refused as malformed.
- The message vocabulary — a status, a method, a request and a list of field
lines, in
src/h3/message.rs. These were thehttpcrate’s until decision D78; carrying it meant every decoded field was validated and copied a second time into aHeaderName/HeaderValue, and the pseudo-headers were folded into a URI that the tunnels then took apart again. A field section is now decoded once, validated once, and carried in one type, and the crate is gone from the dependency tree.
What it leaves out, deliberately:
- The QPACK dynamic table.
SETTINGS_QPACK_MAX_TABLE_CAPACITYandSETTINGS_QPACK_BLOCKED_STREAMSare both advertised as 0, which RFC 9204 §3.2.3 and §2.1.2 make binding on the peer’s encoder. A field line referencing a dynamic entry is therefore a protocol violation answered withQPACK_DECOMPRESSION_FAILED, not a gap in the implementation — and the eviction and head-of-line blocking that go with a dynamic table are gone with it. The peer’s encoder and decoder streams are still read, instruction by instruction: with no table, an insertion, a capacity above zero, a Section Acknowledgment or an Insert Count Increment is the connection error RFC 9204 §3.2.2, §4.3.1, §4.4.1 and §4.4.3 make it, and only a zero capacity and a Stream Cancellation pass. - Huffman encoding. A response here is a status line and at most two short fields; compressing it would save a handful of bytes per tunnel. Decoding is implemented because a client’s request arrives Huffman-coded.
- Server push and WebTransport. A push stream from a client is a connection
error either way (RFC 9114 §6.2.2), and
webtransportis a:protocolthis server answers 501 to. The push bookkeeping frames are still judged: every CANCEL_PUSH names a push this server never promised (RFC 9114 §7.2.3), and a MAX_PUSH_ID may not shrink (§7.2.7) — bothH3_ID_ERROR.
A connection error is a quinn::Connection::close carrying the HTTP/3 code,
which is exactly what RFC 9114 §8 defines one to be. Nothing has to be
propagated between tasks: closing the connection makes every operation on it
fail on its own, so only the reason is recorded on the way past — quinn
overwrites its own stored reason with “closed locally” — and read back by the
accept loop.
Which frame types may appear is decided first, and from the frame header alone.
SETTINGS, GOAWAY, CANCEL_PUSH, MAX_PUSH_ID and PUSH_PROMISE on a request stream
are a connection error of type H3_FRAME_UNEXPECTED whatever length they declare
(RFC 9114 §7.2.3–§7.2.7, and §4.1 for GOAWAY, whose “wrong stream” rule §7.2.6
states for a client only). Once a CONNECT has been answered, RFC 9114 §4.4
narrows the stream to DATA and unknown types, so a HEADERS frame there is a
connection error too. Ordering these ahead of the size checks is what keeps a
431 from being the answer to something that is not a field section, and what
stops a frame the peer was never allowed to send from being charged for.
Field sections are bounded at MAX_FIELD_SECTION_SIZE = 64 KiB, advertised as
SETTINGS_MAX_FIELD_SECTION_SIZE so a peer that respects SETTINGS never sends
more, and enforced on receipt so one that does not gets no further. The frame
layer refuses an oversized non-DATA frame from its declared length, before a
byte is allocated for it.
That bound is per frame, and a peer may open as many streams as its transport
parameters allow. What one connection may hold in frames it has announced but
not finished is therefore bounded separately, at HEADERS_BUFFER_BUDGET = 1 MiB
summed over its request streams: a frame is charged for the length it announced
the moment the decoder commits to buffering it, and the charge is returned when
the frame completes or when the stream carrying it ends. The request that would
take a connection past that is refused with 431 and stopped with
H3_EXCESSIVE_LOAD, which is the same answer a single oversized field section
gets; the connection and its tunnels are untouched. A CONNECT request with
credentials is a few hundred bytes, so every request stream the transport allows
could be mid-HEADERS at once and still be using a tenth of the budget — what
reaches it is sixteen field sections of the largest advertised size in flight
together, and the seventeenth of those is what is refused. Sixteen streams at
once is also the whole of what an unauthenticated connection may hold, so a peer
that has proved nothing can meet this budget and never overshoot it: the 431
belongs to a connection past the credentials check, and against such a
connection’s 1024 streams it is the only thing standing. Only frames that are
legal where they arrived are counted, since the type verdict comes first: a peer
cannot hold the budget with PUSH_PROMISE frames on request streams, nor with
field sections announced on tunnels that may carry none. The peer’s control
stream is not counted in it either: there is one of it per connection, it
buffers one frame at a time, and nothing on it can be refused stream by stream.
One choice here is worth stating, though it is within the RFC: a malformed
request is answered with a bare 400 and a clean stream close rather than a
reset wherever the fault is visible at the tunnel layer – a CONNECT-UDP request
carrying Content-Length or Content-Type (RFC 9297 §3.2), or any request
carrying one of the connection-specific fields RFC 9114 §4.2 forbids
(Proxy-Connection, Keep-Alive, Transfer-Encoding, Upgrade). RFC 9114
§4.1.2 lets a server “send an HTTP response indicating the error prior to
closing or resetting the stream”, and it_tcp and it_udp pin the 400. The
Connection field itself is refused at decode time with H3_MESSAGE_ERROR,
earlier still. The four-field check runs before the credentials check rather
than after it, unlike the routing it precedes: a message carrying one is
malformed whoever sent it, and judging it afterwards answered that MUST with a
407 to an unauthenticated peer and a 400 to an authenticated one.
A peer that closes its QPACK encoder or decoder stream is treated as
H3_CLOSED_CRITICAL_STREAM, the connection error RFC 9204 §4.2 requires, and
both endings count: a clean finish and a reset alike. The one exemption is a
connection that has already ended, where the closure is the connection going
away rather than a fault the peer committed. That is the exemption the control
stream makes for the same race.
HTTP Datagrams are hand-rolled in src/datagram.rs. That started as a way around
h3-datagram 0.0.2, which tagged every datagram with Quarter Stream ID 0 and so
misrouted every session after the first on a connection; it is simply ours now —
about thirty lines of varint work with no dependency behind them.
Why quinn-proto is patched (temporary)
Cargo.toml carries a [patch.crates-io] stanza pointing quinn-proto at a
commit on upstream’s 0.11.x branch. quinn-proto’s MTU black-hole detector treats
an ordinary congestion loss burst during a bulk transfer as proof that the path
has stopped carrying the current MTU, so a download makes the connection fall
back to the 1200-byte floor — and on the 0.11.x branch every further loss burst
at the floor re-triggers the detector and pushes the next probe out again, so the
MTU stays there for as long as the transfer lasts. That branch was also missing
two comparison fixes that upstream had landed on main only.
The pinned commit, 48455d3, is the quinn-proto-0.11.17 tag plus five fixes
(and two unrelated backports the branch picked up in between: ACK frames
bundled into DATAGRAM/STREAM packets, and a minimum-RTT path statistic).
Three are in quinn-proto/src/connection/mtud.rs, brought to 0.11.x by
quinn-rs/quinn#2799:
- “Fix some comparisons in the black hole detector” and “Relax MTU discovery state assertion”, backported from quinn-rs/quinn#2400.
- “proto: treat an equal-size delivery as evidence against a preceding loss
burst”. The actual bug is
#2791 and the fix
#2792, merged to
main.
The fourth is in quinn-proto/src/connection/datagrams.rs: the
outgoing-datagram eviction loop subtracted an evicted datagram’s size from the
buffered-bytes counter twice, so sustained eviction underflows the counter and
ends in a panic inside send_datagram while the connection lock is held. The
bug is #2805 and the fix
#2806, landed on 0.11.x.
The fifth is in quinn-proto/src/endpoint.rs: a datagram carrying an
unsupported QUIC version elicited a Version Negotiation packet whatever its
size, where RFC 9000 Section 5.2.2 says servers MUST drop the small ones. A
spoofed 15-byte probe therefore got a larger reply, an amplification vector
aimed squarely at the port scanner the SNI gate exists to defeat. The fix is
#2822 on main, backported to
0.11.x by #2823.
No 0.11.x release carries these yet, which is why the stanza pins a commit rather than a version. Because that commit’s version is the upstream tag, anything true of quinn-proto 0.11.17 is true here except those five fixes.
Exit condition: drop the stanza as soon as a quinn-proto 0.11.x release
includes the fix, then cargo update -p quinn-proto and remove the CI audit
workaround described below. Two things do not work while it is in place:
- Dependabot does not touch git dependencies, so it cannot bump this one. The
pinned commit is moved by hand whenever quinn-proto ships a security or
hardening release, and moving it is forced if a
quinnrelease raises its quinn-proto requirement above 0.11.17. - cargo-audit skips lockfile entries whose source is not the default registry, so
the patched entry would fall out of RustSec coverage. The
auditjob in.github/workflows/ci.ymlworks around this by scanning a copy ofCargo.lockwith that onesourceline rewritten back to the registry — which is sound precisely because the version is the upstream tag.
The h3api facade
src/h3api.rs is the facade over src/h3: the only module allowed to name a
type from it, so that everything else — conn, quic, the tunnels — sees
bytes and quinn types plus the handful of names it exports. The facade began
as insulation from a 0.0.x dependency’s API churn and was kept on its own terms:
it is the list of what a proxy actually asks of HTTP/3, and it is short enough to
read in one sitting.
The message types — Request, Status, Fields — are re-exported through it
rather than wrapped in something else. The rule the facade enforces is about
where a type is named, not about hiding it: nothing outside src/h3 reaches
into that module, so this one file still says everything the rest of the crate
may assume about HTTP/3.
One gotcha it cannot hide: h3api::Connection::handshake consumes the
quinn::Connection, so clone it first if you also need to send datagrams on
it. Receiving them needs no clone — the HTTP/3 connection routes them itself.
Testing
Integration tests assert on-wire behaviour, not internal state: status codes,
response headers, and in it_settings the actual bytes the server writes on its
control stream. That is what makes a dependency bump reviewable — the tests
describe the protocol, so they still mean something after the implementation
underneath changes.
Two tests are load-bearing beyond their names:
- The multi-session concurrent CONNECT-UDP test is the regression baseline for Quarter-Stream-ID-class bugs. Three sessions on one connection, each verified to receive its own traffic. Never weaken it.
it_migrationrebinds the client endpoint mid-tunnel and asserts that existing tunnels survive the address change and that new ones can still be opened, which pins QUIC connection migration against an accidentalmigration(false)or an upstream regression. What makes migration work behind NAT at all is quinn’s non-zero-length server connection IDs (8 bytes by default) — the property RFC 9308 §9 strongly recommends for exactly this reason — so any future endpoint tuning must not shorten them to zero.
it_stress keeps a heavy tier behind #[ignore] (500 concurrent tunnels, 10000
setup/teardown rounds) and a light tier in the default run that catches slot
leaks by shrinking the quota until a leak turns into an observable 503.
These pages are checked too. it_docs reads docs/ as text and asserts three
things a machine can settle: that the configuration keys documented here and the
ones script/config.example.toml ships are one set, and that the documented set
still parses as a Config; that every constant a page quotes in the
`IDENT` = value notation holds the value the crate compiles; and that every
docs/<page>.md#<anchor> reference in src/, in tests/ and in these pages
themselves names a heading that exists. What no gate can check is whether a
sentence is true, so that half stays a review question.
Test infrastructure lives in tests/common/: an in-process server plus
self-signed certificates from rcgen, so no test needs a fixture on disk, and
the HTTP/3 client in tests/common/h3client.rs that drives it.
One thing that suite cannot do is disagree with itself: that client is built on
the same codec as the server, so a misreading of the framing or of QPACK is one
both ends share. The interop CI job closes that gap by starting a real
volto process and driving it with two clients that share no code, no QUIC
stack and no reading of the RFCs with it or with each other. Go’s
masque-go on quic-go speaks CONNECT-UDP
over the RFC 9298 default URI template, with authentication on and the server’s
certificate trusted rather than skipped; it asserts multi-round echo on one
session, three concurrent sessions on a single QUIC connection receiving only
their own traffic, Proxy-Status on a refusal, and a 407 when credentials are
omitted. Python’s aioquic covers what
masque-go cannot: the plain CONNECT (TCP) tunnel — round trips, two concurrent
tunnels, and the RFC 9114 §4.4 half-close — plus a datagram carrying an unknown
Context ID, which RFC 9298 requires to be dropped silently and which only a
client that writes its own Context ID prefix can send. The clients live in
tests/interop/ (a Go module) and tests/interop/aioquic/ (a Python script),
so cargo test neither sees nor needs them.
A third independent implementation judges the field-section encoding in
particular, and its verdicts are checked in rather than re-run: the differential
oracle in tests/interop/difforacle/ puts generated and mutated field sections
through this server’s QPACK and Huffman decoders and through ls-qpack (via
pylsqpack), comparing both the accept/reject verdict and the decoded fields.
It needs a pinned Python environment and so runs on demand like the fuzz
targets, but what it settled lives in tests/it_diff_oracle.rs and is
re-checked on every cargo test: the five inputs the two implementations
answer differently, none of them a fault here, and ls-qpack’s own copy of the
RFC 9204 static table, entry by entry. That table is the one thing the in-tree
tests cannot check for themselves, since src/h3/qpack.rs transcribes RFC 9204
Appendix A by hand and its unit test can only compare that transcription with
itself.
Every hand-written parser that reads bytes a peer chose also has a
coverage-guided fuzz target under fuzz/, run on demand on a nightly toolchain
in a Linux container rather than in CI – which only type-checks them, since
fuzz/ is a workspace of its own and its targets reach items that are public
only because they ask for them, so a rename in src/ would otherwise break the
fuzzers silently between runs. fuzz/README.md maps each target to the parser
it covers and has the invocation. Beside the five codecs — the frame
state machine, QPACK, Huffman, the capsule parser and the datagram codec — that
means the request validator that reaches RFC 9114 §4.1.2’s malformed verdict,
the two parsers that turn a client-named target into a host and a port, and the
credentials field, which is the one attacker-shaped byte string read before
anything has authenticated. The property tests in tests/it_props.rs and
tests/it_fuzz.rs state the same kinds of invariants on stable Rust, so they
are what CI runs on every push.