What's New
Multi-Provider Data Generation
--provider ollama— generate synthetic data via local Ollama instance (auto-detects at localhost:11434)--provider anthropic— generate via Claude API (ANTHROPIC_API_KEY from env)--provider vllm— generate via local vLLM server--ollama-modelshorthand flag for quick Ollama generation
Domain Templates
--template code— code instruction pairs (Python, JS, Go, Rust, Java × function/debug/explain/refactor/test)--template conversation— multi-turn dialogues (configurable turns, topics)--template qa— question-answer from context documents--template preference— DPO/KTO/ORPO preference data--template reasoning— chain-of-thought / GRPO reasoning data (math/logic/code)
Quality Pipeline
--validate— auto-validate generated data, remove malformed entries--filter— auto-filter by coherence quality scoring--dedup— auto-dedup via MinHash (requiressoup-cli[data])--quality-pipeline— run all three in sequence
Security
- Ollama provider: localhost-only validation (remote Ollama blocked)
- Anthropic provider: API key from environment only, never CLI arg
- vLLM provider: SSRF protection (scheme whitelist + localhost-only HTTP)
- Output path traversal protection (
..blocked in output paths) - Configurable rate limiting (
--requests-per-minute, default: 60)
Stats
- 1669 tests across 59 test files
- 84 new tests in
test_synth_data_pro.py
Install / Upgrade
pip install --upgrade soup-cli