MakazhanAlpamys/Soup v0.13.0

v0.13.0 — Batch Inference + TensorBoard

on GitHub

What's New

Batch Inference (soup infer)

Run a model on a list of prompts and save results to JSONL:

soup infer --model ./output --input prompts.jsonl --output results.jsonl

• Reads JSONL ({"prompt": "..."}) or plain text (one prompt per line)
• Output includes prompt, response, and tokens_generated fields
• Progress bar with throughput summary (tok/s)
• Supports LoRA adapters (auto-detects base model) and full models

TensorBoard Logging (--tensorboard)

Log training metrics to TensorBoard for local visualization:

soup train --config soup.yaml --tensorboard
tensorboard --logdir ./output/runs/

• Mutually exclusive with --wandb (pick one)
• Uses HuggingFace Trainer's built-in TensorBoard integration

Supported Models

Added a supported models table to README covering:

• Llama 4 (Scout, Maverick)
• Gemma 3 (9B, 27B)
• Qwen 2.5/3 (7B, 8B)
• Phi-4 (14B)
• DeepSeek (R1, V3)
• Mistral/Mixtral

Stats

• 906 tests, 44 test files, 56.32% coverage
• All CI checks pass

Install / Upgrade

pip install --upgrade soup-cli