Skip to content

Streams marked closed_for_gc are never disposed until connection close (write_buffer retained) #30

Description

@MKS2508

Summary

Bidi streams that are fully closed (closed_for_gc = true) and fully acknowledged are never queued for disposal — they retain their write_buffer in the StreamsMap until the connection closes. On long-lived connections serving many short-lived streams, this produces per-stream retention that scales with stream count and only drops on connection close.

Context

Observed while building a WebTransport workload on top of event_loop.Server, serving many short-lived bidi streams over a single long-lived connection (on the order of hundreds to thousands of streams per connection). The allocator is wrapped in a tracking allocator for telemetry; live_bytes (instantaneous allocated - freed) grows monotonically with the number of streams served and does not drop until connection close.

Behavior

The intended disposal path:

  • collectClosedStreams (src/quic/stream.zig:1116-1137) marks fully-closed bidi streams with closed_for_gc = true and adjusts flow-control counters, but does not queue them for disposal. The comment at stream.zig:1127-1129 documents this as intentional:

    Delay disposal: keep the stream in the map so PTO can reset send_offset for retransmission under loss. Stream will be cleaned up when the connection closes.

  • The only producer of queueDisposal is the STREAM+FIN handler in connection.zig:1904, guarded by !strm.closed_for_gc.

The guard means that once collectClosedStreams has marked a stream, the STREAM+FIN disposal path can no longer fire for it. So a fully-closed, fully-ACKed stream stays in the map (with its write_buffer) until Connection.deinit runs at connection close. StreamsMap.deinit does free everything, so nothing is truly lost — but for any connection that stays open and keeps serving new streams, the retention is unbounded in stream count.

Impact

On our workload (~500–1000 bidi streams served over a single 90s connection, each stream's write_buffer retained after close), this manifested as ~800 MB retained for the connection's lifetime, reclaimed cleanly on close. Not a leak in the strict sense — but it behaves like one for long-lived multi-stream connections.

Related to (but distinct from) #25 — both touch the boundary between stream lifetime and connection lifetime. The proposed fix is careful to preserve the PTO property the "Delay disposal" comment is protecting: once the send side is fully acknowledged, PTO has nothing to reset.

Proposed fix

Opened #29 with a minimal change: move the disposal check to onAck processing, so a closed_for_gc stream is queued for disposal at the moment its last byte is acknowledged, under the same guard the existing disposal path already relies on:

try s.send.onAck(sf.offset, sf.length);
if (s.closed_for_gc and s.send.retransmit_count == 0 and !s.send.hasUnackedData()) {
    self.streams.queueDisposal(s.stream_id);
}

retransmit_count == 0 and !s.send.hasUnackedData() means there is no un-ACKed data and no in-flight retransmission, so PTO has nothing to retransmit — send_offset is fully settled. The "Delay disposal" concern (PTO resetting send_offset under loss) does not apply once the send side is fully ACKed.

#29 also adds lifecycle tests for collectClosedStreams, drainDisposalQueue, queueDisposal, and closed_for_gc, which currently have no test coverage in the codebase.

No pressure to take it as-is — happy to rework into a collectClosedStreams-adjacent pass or any other shape that fits the codebase better. Feedback very welcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions