valnesfjord/tg-ws-proxy-rs v2.2.5-beta.1

tg-ws-proxy-rs v2.2.5-beta.1

on GitHub

Lower memory use and faster connection handling

This release addresses the memory growth reported in
#97 and
#113. The
leak-shaped growth was not an unbounded Rust collection: most of it came from
memory reserved and retained independently by every active connection. A busy
proxy multiplied large async state machines, WebSocket buffers, duplicate
MTProto packet storage, short-lived strings, and relay tasks by the number of
connected clients.

The optimization keeps the connection pool and media-specific warm sockets.
Those are important for media latency and have not been disabled, merged, or
made smaller. Instead, this release reduces the cost of each live client and
makes pool maintenance more efficient.

Smaller session state

• Route selection now runs in a short-lived boxed future. Once an upstream
  route wins, synchronous dispatch creates only the selected WebSocket,
  upstream MTProto, or raw-TCP bridge. The complete fallback ladder is freed
  before the session-long relay await instead of remaining embedded in every
  client future.
• A regression test caps the outer per-client future at 4 KiB. On x86-64 its
  size fell from 20,784 to 1,808 bytes, a 91.3% reduction in the fixed async
  state allocated for every accepted client.
• Client addresses remain allocation-free SocketAddr values through routing
  and bridging. Direct-pool lookup borrows the configured DC IP, and only a
  refill that actually reserves a bucket creates an owned target string. A
  1,000-request burst no longer creates 999 throwaway copies for refill tasks
  that have no work to do.

The complete release, compared with main, reduces peak private memory from
257.9 MiB to 118.1 MiB for 1,000 held direct-WebSocket clients (-54.2%). Proxy
CPU time in the same workload falls from 1.53 s to 1.19 s (-22.2%).

Bounded WebSocket buffering and single-pass MTProto framing

• Incoming WebSocket frames and messages are limited to 4 MiB instead of
  tungstenite's 16 MiB frame and 64 MiB message defaults. Telegram media parts
  are at most 1 MiB, leaving protocol headroom without allowing one unusual
  upstream frame to reserve tens of megabytes for a connection.
• The WebSocket write-buffer target is 16 KiB instead of 128 KiB. Every bridge
  send is awaited and flushed, so the larger target did not provide useful
  batching in this proxy.
• MTProto WebSocket framing now reads plaintext packet-length headers before
  relay encryption. The splitter no longer decrypts the complete upload a
  second time and no longer retains parallel plaintext and ciphertext buffers.
  Once a packet is complete, its encrypted allocation moves directly into the
  WebSocket message.
• Cloudflare Worker remains deliberately different: it tunnels a raw TCP byte
  stream and does not packet-align WebSocket messages. This preserves uploads
  whose MTProto packets exceed Cloudflare's 1 MiB WebSocket-message limit.

In the final Linux comparison, 64 simultaneous direct-WebSocket media clients,
each sending eight 1 MiB packets plus two smaller packets in both directions,
reduced peak private memory from 308.2 MiB to 221.2 MiB (-28.2%) and proxy CPU
from 3.57 s to 3.06 s (-14.3%). Every returned byte was checked.

Cheaper full-duplex relays

• Traffic counters accumulate in task-local u64 values and publish once when
  a direction finishes or is cancelled. This removes a mutex acquisition from
  every relay chunk while preserving accurate partial totals on 32-bit targets
  where 64-bit atomics are unavailable.
• TCP streams use Tokio's specialized owned read/write halves. This removes the
  shared mutex used by generic tokio::io::split from inbound clients,
  upstream MTProto proxies, TCP fallback, and the connectivity checker.
• A bridge now downloads in the existing client task and spawns only the
  upload direction. This removes one Tokio task per active session while
  preserving simultaneous upload/download, WebSocket backpressure, partial
  traffic totals, and immediate cancellation when either peer closes.

Incremental testing of the owned TCP halves reduced direct-WebSocket proxy CPU
by 2.7% and raw-TCP wall time by 2.1%. Removing the second bridge task reduced
held raw-TCP memory peaks by 3.5%. In the cumulative final comparison, 1,000
held TCP-fallback clients use 10.7% less private memory. Under a heavy TCP
media transfer, private memory is 7.0% lower; the 0.7% CPU and 2.2% wall-time
differences are within run-to-run noise.

Parsed once, borrowed per connection

• Immutable inbound and upstream MTProto secrets are decoded once and stored
  in ready-to-use binary form. Client handshakes borrow cached keys and FakeTLS
  hostnames instead of repeating hex decoding and allocations.
• Standard and configured NO_PROXY rules are parsed once. Matching uses
  borrowed host/port values instead of constructing and parsing a URL for every
  outbound connection.
• Direct TCP, SOCKS5, SOCKS5h, and ordinary HTTP CONNECT paths keep target
  hostnames borrowed unless a protocol frame must own the bytes.
• FakeTLS handshakes build their output in one allocation, read the inbound
  initialization into an exact-size record, and consume coalesced pending data
  by offset rather than repeatedly moving the remaining bytes.

With 1,000 held upstream-FakeTLS clients, the final build uses 48.9 MiB of
private memory instead of 59.8 MiB (-18.2%). With inbound FakeTLS and 64 extra
configured secrets, private memory falls from 260.6 MiB to 117.6 MiB (-54.9%)
and CPU from 1.33 s to 1.08 s (-18.8%).

Pool latency without a larger pool

• Direct and Cloudflare refills reserve capacity in their bucket before
  spawning. A 1,000-client burst against one bucket now creates one refill
  task instead of 1,000 tasks of which 999 wake up only to discover that the
  bucket is already full.
• Direct-pool fill uses tightly bounded concurrency: two handshakes per bucket
  and two startup buckets, for at most four in-flight handshakes regardless of
  configured pool size. This avoids an unbounded TLS/DNS/socket burst.
• Normal and media pools remain separate, --pool-size keeps the same meaning,
  and all expected spare connections are still created and replenished.

With a deterministic 250 ms handshake delay, the median time to prepare four
spares fell from 1,138 ms to 256 ms (-77.5%) over seven interleaved Linux runs.
Peak private memory during warmup fell from 5.57 MiB to 5.31 MiB (-4.6%); the
steady-state number of sockets is unchanged.

Lazy Cloudflare fallback

• Cloudflare fallback creates only the hostname for the attempt it is about to
  make. It no longer builds a 2 * N owned-string queue and string-keyed
  deduplication set for every client.
• Balanced domains rotate as a borrowed cyclic view rather than cloning the
  complete domain list. The fallback still preserves media-first ordering and
  the intentional kwsN-1 to kwsN retry.

The incremental 65-domain/1,000-client benchmark reduced CPU by 8.2%, peak RSS
by 3.8%, and private memory by 3.9%. Across all optimizations in this release,
the same Cloudflare workload reduces private memory from 264.3 MiB to 118.3 MiB
(-55.2%) and CPU from 1.38 s to 1.09 s (-21.0%).

Cumulative Linux performance

The following table compares the v2.2.4 main baseline at 641c6b6 with the
optimized v2.2.5 code. Both binaries were built with Rust 1.97.1 using the same
release profile and Debian Bookworm container. Each workload used five
interleaved runs; the table reports medians. Memory is sampled from Linux
/proc/<pid>/smaps_rollup, and CPU time from /proc/<pid>/stat.

Held-session wall time includes a fixed backend hold and test-process startup,
so differences around 2-5% there are scheduler noise rather than connection
throughput. Likewise, the small TCP/Worker CPU changes are not claimed as
regressions: the Worker workload completed 3.1% faster while using 3.2% more
accounted CPU. The stable result across every workload is lower peak memory;
the direct WS and Cloudflare hot paths also show a clear CPU reduction.

The stripped x86-64 release binary is also 8,728 bytes smaller than main
(4,367,848 -> 4,359,120 bytes, -0.2%).

Comparison with v1.7.1

The loopback harness also compared the final build with v1.7.1 using fake
Telegram TLS/WebSocket and HTTP CONNECT endpoints. It waits for every upstream
handshake and verifies that every expected transfer drains before sampling the
result. In identical Linux containers with pooling disabled, two runs in
reversed order produced:

The default --pool-size 4 was measured separately. Both versions retained all
16 expected direct-WebSocket spares: four normal and four media connections for
each of DC 2 and DC 4.

The post-drain result includes allocator retention and is not a best-case live
heap snapshot. These workloads therefore report process RSS rather than an
allocation profiler's narrower live-byte count.

Functional verification

Both the baseline and optimized Linux release binaries passed the same
byte-exact synthetic E2E matrix:

• direct WebSocket and direct media;
• Cloudflare proxy and Cloudflare Worker media;
• TCP fallback;
• upstream MTProto and upstream FakeTLS;
• inbound FakeTLS;
• abridged, intermediate, and padded-intermediate MTProto transports;
• concurrent media transfers;
• rejected clients, upstream disconnects, and fail-closed behavior;
• normal/media pool separation and refill.

The functional matrix transfers a 1,065,484-byte mixed payload through every
route. The heavier performance workload moves approximately 516 MiB in each
direction across 64 concurrent clients. Payloads are generated independently
for upload and download and compared byte-for-byte after the proxy.

Release and OpenWrt packaging

• Cargo.toml is now the single source for the stable LuCI package version.
  The OpenWrt builder reads it automatically and supplies it to every SDK make
  invocation; the package recipe no longer carries a version fallback that can
  drift during a release bump.
• A beta tag such as v2.2.5-beta.1 reuses this base-version release-notes
  file. Release CI creates a temporary changelog with the beta heading before
  passing it to parse-changelog, so a duplicate
  docs/release-notes/2.2.5-beta.1.md is not required.
• Explicit package versions remain supported for the APK 2.2.5_beta1 and IPK
  2.2.5~beta.1 formats generated by beta release CI.

Deliberately unchanged behavior

• Pool caching and the default pool size are unchanged. The release reduces
  refill overhead and warmup latency rather than trading away ready media
  connections.
• The global pool mutex remains. Per-bucket mutexes, fixed async shards, and a
  synchronous mutex were prototyped, but Linux A/B results were neutral,
  unstable, or regressive. The existing critical sections are short, so adding
  idle allocations and complexity had no demonstrated benefit.
• Relay buffers remain deliberately small. Router deployments benefit more
  from a low per-connection footprint than from avoiding a few extra syscalls.

Compatibility cost

An upstream WebSocket frame or fragmented message above 4 MiB now closes that
upstream connection instead of allocating up to the old limit. This is well
above Telegram's observed 1 MiB media-part ceiling, but a non-Telegram custom
upstream sending larger WebSocket messages is no longer accepted. Cloudflare
Worker uploads are unaffected because that route uses raw-stream framing.