The Better Together Release
This release gets llama-swap + tailcat ready for use. If you haven't heard of tailcat it makes it easy to share models hosted by llama-swap with your friends or anyone over the Internet. Tailcat is "Tailscale, without Tailscale by Tailscale", which means you get Tailscale's security and connectivity magic without all the extra login and configuration overhead.
Here's how it looks:
With the token anyone can connect to your server securely and directly. No port forwarding, sharing IP addresses or setting firewall rules!
Multiple llama-swap instances can be connected together as peer configurations also support tailcat! An easy to share peer configuration is automatically generated to share with friends:
To get started generate your custom instructions with the built in Docs Agent:
I would love to hear your feedback and how you're using this cool new feature!
Changelog
- 6c7f958 docker/unified: add cuda13 with multi-platform support for arm64 (#1093)
- 82ccfbf filters: set-if-undefined params via "?" key suffix (#1075)
- 41f1431 ui/tailcat: mask connection token, add shareable peer config (#1091)
- 8365956 ui/help: raise Docs Agent max_tokens from 4096 to 65536 (#1089)