Highlights
⛰️ Stability. This release improves stability and performance of the Petals DHT in presence of many servers joined via NAT traversal & relays. Now, the DHT prefers to store the keys on directly reachable peers, so that all peers can access them faster and with less failures. Also, this release contains a minor fix to the block reassignment algorithm that decreases excess reassignments that were leading to the swarm downtime in the past.
🌎 Basic routing. We have improved the routing algorithm for inference, so that clients weakly prefer servers holding more blocks to minimize latency and increase inference speed. This is only a basic algorithm, and we are working on smarter routing (taking into account latency, throughput, etc.) for both inference and fine-tuning in future releases. This release also makes the servers share more technical information about themselves (their version, free cache, etc.), so it can be used by the smarter routing algorithms in future and shown at http://health.petals.ml for debugging purposes.
What's Changed
- Fix fine-tuning notebooks intros by @borzunov in #194
- Ignore network RPS if we failed to measure it by @borzunov in #198
- Make client ignore blacklist if all servers holding a block are blacklisted by @borzunov in #197
- Increase tolerances in test_tp_block by @justheuristic in #196
- Fix --no_auto_relay help by @borzunov in #199
- Use length-weighted sampling in routing for inference by @justheuristic in #204
- Return available cache size in rpc_info() by @justheuristic in #191
- Add service checking direct reachability from peers by @justheuristic in #195
- Report server version and dht.client_mode in rpc_info(), check for updates on startup by @borzunov in #209
- Don't switch blocks if it makes swarm disjoint by @borzunov in #210
- Fix output shape when resuming generation by @borzunov in #211
- Improve errors in case of missing blocks, suggest to join your own server by @borzunov in #212
- CI: Convert model only when convert_model.py or setup.cfg change by @borzunov in #213
- CI: Update deprecated actions, don't measure network RPS by @borzunov in #215
- Bump version to 1.1.1 by @borzunov in #214
Full Changelog: v1.1.0...v1.1.1