Ohh, support for the v1/realtime API with LFM3 voice chat in the Playground!
The Help agent has been updated with a guide on how to quickly get it set up
and working.
Also in this release are a few compatbility and quality improvements.
Thanks to @noctrex, @SriRamanujam, and @ltjax for the contributions.
- PR #1278 Feat/sidebar category submenus: adds category submenus to the left sidebar menu, mirroring the capability-derived categories recently added to the Models list by @noctrex and @mostlygeek
- PR #1288 Configurable token rate sources: adds hints, docs and usage extraction for hipfire so token/s stats are captured correctly when running hipfire behind llama-swap, as part of the pluggable per-upstream usage parsing by @ltjax
- PR #1291 cmd/kubeswap: Support defining annotations and service accounts: allows defining annotations on generated deployments and pods (rejecting the reserved
kubeswap.io/model-idkey) and setting.spec.template.spec.serviceAccountNameon the Deployment, without managing the ServiceAccount itself by @SriRamanujam - PR #1297 fix(logmon): stop memory accumulating across config reloads: closes a process's dedicated log monitor when
ProcessCommand.runreturns so discarded router generations no longer leak memory, because each monitor's surviving broadcast goroutine kept its 100 KiB history buffer reachable after a hot reload (129 MiB retained after 100 router replacements in testing, now ~0) by @junmo-kim - PR #1298 makefile: use .stamp for UI dependency checks: uses a
.stampfile to detect UI dependency changes, fixing a build error after pulling the latest version when UI dependencies changed by @mostlygeek - PR #1299 internal/server: add /v1/realtime proxying and a Realtime playground: proxies OpenAI Realtime API style websockets to the upstream named by the
?model=query and adds a Playground > Realtime tab for speech-to-speech models like LFM3 Audio through llama-server (push-to-talk or hands-free voice detection, streamed playback, replay/download per turn), records each session as one Activity row with reply, token, decode-rate and audio-second metrics, detects audio output from/v1/modelsoutput_modalities, and adds a KB guide with llama-server configuration examples by @mostlygeek