SikamikanikoBG/homelab-monitor v0.26.0

v0.26.0 — The LLM Benchmark Lab: measure what your models actually do on your GPU

on GitHub

"Will it fit, and at what context?" stops being guesswork. The new Benchmark Lab loads your local models and measures what they actually do on your GPUs — generation & prompt tokens/sec, load time, the VRAM↔RAM split, and the largest context that still fits fully in VRAM — per card, priced by the same tariff engine as the rest of the app, and stored so you only re-run when something changes. Plus an external contribution: unified-memory APUs now report their real usable VRAM (GTT), not the 512 MB carve-out.

Changed

• Benchmark Lab results are now a compact, filterable, sortable table instead of cards. Cards don't scale — with dozens of runs they're hard to scan. The results are now one table: one row per run with the model, setup (which GPU), gen/prompt tokens/sec (with an inline speed bar), load time, fit, max-VRAM/recommended context, the VRAM↔RAM split, energy and cost, and when it ran. Click any column to sort, type in the filter box to narrow by model/setup/fit, click a row to open its context sweep, and tick rows to overlay them in Compare. Running/failed rows show their status inline; the "weights spread onto a smaller card" hint moves to a ⚠ on the Fit cell (and a banner in the sweep).

Fixed

• Benchmark Lab: a too-big context no longer hammers the box, and progress keeps reporting across tab switches. Two issues from real use: (1) sweeping ascending contexts kept trying ever-larger sizes even after one had already failed to fit — so a 30B model under memory pressure would OOM (ollama HTTP 500) on 64k, then 128k, then 256k in a row. The sweep now stops at the first context that can't be allocated (larger ones can only fail too), and the opaque "HTTP 500" is reported as "context too large for available memory — ollama couldn't allocate the KV cache", with the sweep chart calling out the ceiling. (2) The running-job progress didn't visibly advance after switching tabs or reopening the dashboard: the poll is now independent of which tab is showing (so it keeps ticking and is current the instant you return), shows elapsed time, survives a render error, and no longer flickers the charts each tick. The per-generate timeout is also capped lower so a single stuck generation can't hold the single-flight slot for long.
• Reloading the dashboard while on the Benchmark Lab no longer leaves a dead page. Opening the dashboard with #benchmarks in the URL — or with the Lab as your last-used tab — threw a ReferenceError during init (the Lab's renderer lives in a later script block than the tab switcher) and killed event wiring and the poll loop before they started. The call is now guarded and the Lab renders itself when its block loads. (#245)
• Unified-memory APUs now report their real usable VRAM, not the 512 MB carve-out. On AMD APUs (a Ryzen iGPU driving a mini-PC, say) the dedicated "VRAM" is just a small carve-out — the GPU actually works out of shared system memory (GTT). The amdgpu reader (hub and remote probe) now reports the GTT pool when the card looks like an APU (≤ 1 GiB dedicated VRAM next to a large GTT pool — think Ryzen AI Max / Strix Halo), so an APU shows its true capacity instead of a permanently-full 512 MB; discrete cards keep reporting VRAM exactly as before. (contributed by @andreahaku, #237)

Added

• Benchmark Lab: pick which GPU(s) to benchmark on, see the setup in every result, and overlay runs to compare. Three additions to the Lab: (1) GPU device selection — since ollama has no per-request device choice, choosing a card spins up a throwaway ollama container pinned to exactly those GPU(s) (reusing your existing models volume, on a scratch port), runs the sweep, and tears it down — your main ollama is never touched. This is how you answer "all VRAM vs which one": benchmark the same model on each card and see. (Needs controls enabled, since it launches a container.) (2) The setup is recorded and shown on every leaderboard row and card (e.g. "⚙ RTX 3090" / "Quadro P2000"). (3) A Compare view overlays any set of stored runs on one chart — tokens/sec and VRAM across context — so e.g. the same model on a small card vs a big one sits side by side.
• Benchmark Lab — a new AI tab that measures what your local models actually do on your GPU, and remembers it. Pick one or more ollama models and a set of context sizes; the Lab loads each model and runs a short generation across the ladder, recording generation & prompt tokens/sec, load time, the VRAM↔RAM split (how much spilled to system RAM, straight from ollama's size vs size_vram), the largest context that still fits fully in VRAM — the recommended cap — a fit verdict (fits VRAM / spills to RAM / CPU), and the power, energy and cost of the run (priced by the same tariff engine as the rest of the app). On a multi-GPU box it also attributes which card the weights landed on and warns when a model spreads onto a smaller/slower secondary card. Results are stored so you only re-run when something changes, with a one-click re-run; the tab shows a speed leaderboard, per-model cards, and a context-sweep chart (tokens/sec and VRAM vs context). Benchmarking is an active, opt-in operation (it loads models and runs inference) and is single-flight so it never stampedes the GPU. New endpoints GET/POST /api/bench, GET /api/bench/<id>, GET /api/bench/targets, POST /api/bench/cancel, DELETE /api/bench/<id>; new MCP tools get_benchmarks / get_benchmark. Off automatically when no ollama endpoint is reachable (BENCH_ENABLED, COPILOT_OLLAMA_URL).

Upgrade: hit Update in the dashboard, or docker compose pull && docker compose up -d. Pinned image: sikamikaniko123/homelab-monitor:0.26.0.