Major new rqbit release 9.0.0-beta.0

The are some major new features and some breaking changes in the API / CLI.

TLDR

• rqbit now supports uTP protocol, which will help discovering more peers. Enable via --experimental-enable-utp-listen. I'll probably enable it by default once 9.0.0 full is released.
• breaking CLI change: you need to replace --tcp-min(max)-port with --listen-port if you used those.
• breaking librqbit changes: some SessionOptions are moved around

uTP support

After a long wait, we finally have support for uTP (uTorrent transport protocol) done through specially created librqbit-utp library.

What does that mean for end users? Mostly, that you'll get more peer connections, especially on torrents with low seed count. It turns out that in the wild there's lots of peers that only have uTP enabled, but they don't listen on TCP. For those torrents previously rqbit couldn't connect to those peers at all.

This turned out to be one of the hardest pieces of code I've ever written, it's pretty much as complex as creating a TCP stack from scratch.

There are some caveats though, which is why uTP is disabled by default and you need to enable it explicitly via --experimental-enable-utp-listen.

uTP caveats

First, the original reason for uTP creation was LEDBAT congestion control algorithm, which is notably missing from the current implementation, and we use CUBIC (the modern TCP congestion control algorithm) instead. I'll probably add LEDBAT someday, it shouldn't be hard. The reasoning behind LEDBAT is to move torrent traffic to background so that it doesn't interfere with normal network operation on the machine. In practice, in my network conditions this is never an issue: I never notice rqbit slowing down my internet to make it noticeable.

Second, the librqbit-utp code is complex and there might be bugs. It seems stable for me, hence I'm making this release: I was running my rqbit for a while in uTP-only mode. But I'm asking all users to please try it out and report issues.

Third, although I tried hard optimizing it, it can't beat kernel's TCP stacks that were being optimized for tens of years by very talented engineers. TCP runs in the kernel, while uTP in tokio userspace. For this reason, rqbit first tries connecting over TCP, and only if it doesn't work after 1 second, a uTP connection attempt is made.

uTP can saturate my gigabit connection however on Apple devices, so it's not slow, just slower than TCP at the limits, and also noticeable on slower devices. E.g. my OpenWrt router can squeeze ~50 megabytes/s with TCP, but only about 20-25 over uTP. Of course it can be optimized further and it's just the beginning.

One more caveat is UDP socket read buffer is shared among all connections, unlike TCP (which is per connection). librqbit-utp tries to increase SO_RCVBUF, but it's likely to fail increasing it enough at least on Linux. This can result in lost incoming packets which will make uTP work considerably worse with fast download speeds, so to prevent this you might need to increase /proc/sys/net/rmem_max on Linux.

Prometheus metrics exporter

While creating librqbit-utp it proved extremely handy to monitor it using metrics, as logs would be too verbose and have too much overhead.

So I added prometheus metrics available under /metrics HTTP endpoint. For now there's only uTP metrics, but we can add more to rqbit itself.

Other changes including breaking

• added --listen-port, --listen-ip, --enable-prometheus-exporter options
• breaking: env / CLI: RQBIT_TCP_LISTEN_MIN_PORT (--tcp-min-port) and RQBIT_TCP_LISTEN_MAX_PORT (--tcp-max-port) moved to RQBIT_LISTEN_PORT
• breaking: librqbit SessionOptions: peer_opts and socks_proxy_url moved to opts: Option<ConnectionOptions>
• breaking: librqbit SessionOptions: enable_upnp_port_forwarding moved to listen: Option<ListenerOptions>

What's Changed

• Use Rust edition 2024 by @ikatson in #377
• librqbit: backoff crate (unmaintained) -> backon by @ikatson in #378
• uTP support (disabled by default) by @ikatson in #322
• Preparing 9.0.0-beta.0 release by @ikatson in #379

Full Changelog: v8.1.0...v9.0.0-beta.0