kossakovsky/selfhost-ai v1.8.2

on GitHub

Added

• Ollama — Two more runtime knobs are configurable via .env: OLLAMA_SCHED_SPREAD (set to 1 to spread every model across all GPUs instead of packing each onto one GPU — packing can strand free VRAM on a multi-GPU host until a model fits nowhere and falls back to slow CPU/GPU hybrid execution) and OLLAMA_KEEP_ALIVE (how long an idle model stays in VRAM, e.g. 20m, for faster switching between frequently used models). Both default to empty, so Ollama's stock behavior (spread off, keep-alive 5m) is unchanged for existing installs (#102).

Upgrade

make update

Both variables are added to your .env automatically as empty values, so nothing changes until you opt in. On a multi-GPU host, set OLLAMA_SCHED_SPREAD=1 (and optionally OLLAMA_KEEP_ALIVE=20m) in .env, then run make restart.

Two upstream parsing quirks are worth knowing before you edit .env by hand:

• OLLAMA_SCHED_SPREAD — use 1 or 0 only. Any other non-empty value is treated as enabled, so off or no would turn it on rather than off.
• OLLAMA_KEEP_ALIVE — always include a unit. A bare 30 means 30 seconds, not minutes; an unparseable value silently falls back to the 5m default, and a negative value keeps models loaded indefinitely.

OLLAMA_SCHED_SPREAD is a no-op on single-GPU and CPU hosts.

Full Changelog: v1.8.1...v1.8.2