SikamikanikoBG/homelab-monitor v0.28.0

v0.28.0 — Where your VRAM actually went: weights vs context, who's holding the model, and AMD cards that report like NVIDIA ones

on GitHub

A loaded model stops being one opaque number. You can now see the split — weights vs context/KV — how much spilled into system RAM, how many of its runs spilled, and which app is actually driving it. The AI Models tab refreshes in seconds instead of half a minute. And AMD boxes reach NVIDIA parity: per-process VRAM read straight from the kernel's own DRM fdinfo nodes, real card names, clocks, perf level, power cap and memory temperature. Half of this release came from outside the repo — thank you @andreahaku, twice over. It also folds in the mcp SDK pin that was bumped to 0.27.1 but never actually released.

Added

• A model that spills into system RAM is now visible everywhere — with how often it happens and which app is driving it. A model that doesn't fully fit VRAM quietly runs its overflow layers on the CPU; outside the Benchmark Lab that was invisible. The ollama probe now keeps /api/ps size next to size_vram (the difference is the spill), so the AI Models tab shows a Loaded · RAM spill badge and the live VRAM + RAM split, a Runs column counting contiguous load sessions and how many of them spilled, and per-model Used-by pills naming the callers — attributed by time-overlap of the caller↔server connection samples with the model's residency, which finally answers "which app is holding this model" when one server hosts several. The GPU tab gains a warning banner while something is spilling right now, the Insight Feed logs it, and a homelab_model_ram_spill_mb{server,model} gauge exports it. A fully-CPU model (size_vram=0) now correctly reads Loaded instead of collapsing to Idle. (#243)
• AI Models: a loaded model's memory is now split into weights vs context/KV — the answer to "why did it spill into RAM". The Now cell gains a stacked split bar and caption (e.g. "weights 17.3 GB · ctx 3.4 GB"): weights are the model file itself (ollama /api/tags size), and the rest of the residency is the context/KV cache + buffers — the part that grows with the context window. The metadata chips now show the runtime context the load is actually running with (@ 32K ctx, from ollama's /api/ps context_length, model max in the tooltip), and the RAM-spill insight explains the split in words — including that a smaller context window shrinks the KV cache and may fit VRAM. No DB change; older ollama servers simply omit the runtime-ctx chip. (#244)
• AI Models tab refreshes in seconds now. A new light GET /api/ai/now endpoint (no DB, no lock) re-probes just the ollama servers on demand — throttled server-side to once per ~3s — and the tab polls it every 5s while visible (local host, auto-refresh on). Loads, unloads, spill and ctx changes show up in a few seconds instead of the previous ~25s worst case (10s sampler + 15s global poll). (#244)
• AMD per-process VRAM is now read from the DRM fdinfo nodes, so "VRAM by service" names real services on AMD boxes. amdgpu has no --query-compute-apps equivalent, but the kernel already publishes per-client residency in /proc/<pid>/fdinfo/<fd> — so the GPU tab goes from one anonymous system/other band to a row per container, and "Services on the GPU" fills in. Measured on a Ryzen AI Max (Strix Halo, 124 GiB GTT): 99.9% of the kernel's own reported GTT attributed, in 8 ms across 550 processes / 1621 fds. Correctness the naive version misses: a bare fdinfo number is bytes not KiB, fd identity comes from the DRM device number rather than the /dev/dri/ path (so a container that remaps its render node keeps its attribution), and drm-shared-* is subtracted so one dma-buf isn't billed to two clients at once. (contributed by @andreahaku, #247)
• The GPU panel reaches NVIDIA parity on AMD cards — clocks, perf level, power cap, mem-bandwidth and memory temp. All read from amdgpu sysfs in the same per-card pass (starred pp_dpm_* rows, mem_busy_percent, hwmon power1_cap, power_dpm_force_performance_level, the hwmon channel labelled mem), each field optional so a card that lacks one simply hides that chip. APUs expose no product_name, so a Radeon 8060S that used to read "AMD GPU 1" now gets its real name from the host's pci.ids. Rode along: the pooled power chip could read over 100% on a hybrid box, because total draw across both vendors was being compared against the NVIDIA-only cap — the aggregate is now built across vendors, and publishes a cap only when every card actually reported one. (contributed by @andreahaku, #254)

Fixed

• The built-in MCP server no longer crash-loops after a rebuild. The dependency was pinned as an open-ended mcp>=1.9.0, so any rebuild after the SDK's 2.0.0 release pulled in a version that had removed the legacy mcp.server.fastmcp module the server imports — ModuleNotFoundError on import, in a restart loop. Pinned to mcp>=1.9.0,<2. This shipped as a 0.27.1 version bump that was never tagged or published, so it reaches everyone here for the first time. (#259)

Upgrade: hit Update in the dashboard, or docker compose pull && docker compose up -d. Pinned image: sikamikaniko123/homelab-monitor:0.28.0.