mostlygeek/llama-swap v243

on GitHub

Spicy maintainer notes:

v243 is a quality of life release.

1. cmd/vllm-wrapper helps vLLM users avoid the long load times by communicating with an already running vLLM to load and sleep a model via its API. It hijacks the model.cmd and model.cmdStop parameters to send the commands. Neat. Thanks @depohmel for the contribution!
2. Matrix sets can finally use full model IDs and using var is now optional. Using var was annoying for users who ultimately ran face first into a "WTF is this naming limitation" moment. Even this frozen heart is melted by enough user tears. I'll work on lowering that threshold (maybe).
3. #955 makes macros work where they are expected to (hopefully) in the configuration. The new approach does a multi-pass expansion over the yaml source. When there are no more macros to expand the configuration is unmarshalled into the typed configuration structure. Hopefully this is macro's final evolved form. I consider this pokémon fully evolved.

Changelog

• 4c50064 internal/config: relax matrix model reference constraints (#957)
• 4bffcfa cmd/vllm-wrapper: add helper for sleep/wake (#941)
• fa607fe internal/config: refactor macro expansion (#955)