A couple of small improvements:
-
support serving llama-swap behind a path on a reverse proxy. Helpful if you want to front llama-swap with nginx and serve it under a specific path.
-
add support for manually defining voices for text to speech models
-
a small kubeswap update
-
PR #1228 ui,server: serve the UI under a reverse proxy base path: makes the browser side base-path agnostic (relative asset, redirect, favicon and PWA URLs plus an
apiUrl()helper that re-attaches a stripped prefix) so llama-swap works behind a plain prefix-stripping proxy athttps://host/subpath/llama-swap/without routing/ui,/apiand/v1at the proxy root; includes an nginx and Caddy guide (#1227) by @mostlygeek -
PR #1225 Add voices override for speech models: adds a
models.capabilities.voicescapability listing the voice ids an upstream server supports, because the non-standardGET /v1/audio/voicesendpoint returns unreliable responses; falls back to probing that endpoint on a best-effort basis when the list is not set (#1207) by @mostlygeek -
PR #1224 cmd/kubeswap: attach DRA ResourceClaims and mount NFS volumes: adds a repeatable
--resource-claim name=claimflag so backends can get GPUs via the NVIDIA DRA driver (kubeswap only references the claim, pods sharing a claim share its device) and annfs:server/export:path[:ro]kind for--volumeso models can be mounted from NFS without a static PV/PVC per export (#1222) by @gabrielcosi