ROCm/FastFlowLM v0.9.9 on GitHub

FastFlowLM v0.9.9 introduces a major runtime overhaul with enhanced flexibility, dynamic context length control, and full OpenAI-compatible API support.

✨ What’s New

1. 🔗 Unified Model–ViewModel Framework

All models now share the same invoke API.
Easier expansion for upcoming features and model types (ASR, TTS, embeddings, etc.).
Model-specific chat templates and parameters (temp, top_k, top_p, etc.) are now supported.

2. ⏳ Launch-Time Context Length (`ctx-len`)

Users can now set context length at launch in both CLI and server mode with ease.

CLI mode:

flm run llama3.2:1b --ctx-len 8192

Server mode:

flm serve llama3.2:1b --ctx-len 8192

Minimal RAM usage can be as low as 2 GB for small models.
⚠️ Values below 512 are automatically adjusted to 512.

3. 🔌 OpenAI-Compatible API (with Image Support!)

/v1/models endpoint now supported for model listing and fetching.
/v1/chat/completions now accepts image inputs via the "images" field.
Works seamlessly with Open WebUI, LangChain, and other OpenAI-compatible tools.
⚠️ Currently, all APIs (including Ollama) only accept images in BASE64-encoded format.

4. 💻 CLI & Help Improvements

Simplified parameter names for a cleaner CLI:

Old Name	New Name
`temperature`	`temp`
`repetition_penalty`	`rep-pen`
`frequency_penalty`	`freq-pen`
`system_prompt`	`sys-msg`
`context_length`	`ctx-len`
`generate_limit`	`gen-lim`

5. ⚙️ Boost.ProgramOptions Integration

Replaced manual parsing with Boost.ProgramOptions.
Provides cleaner CLI UX, stronger validation, and clearer error reporting.
flm help now shows updated flag names and real-time settings.

🌟 Summary

This release rebuilds the FLM runtime for future-proof extensibility, improved developer experience, and seamless integration with OpenAI-based ecosystems.

ROCm/FastFlowLM v0.9.9 🚀 FastFlowLM v0.9.9 — New Runtime & OpenAI API Support on GitHub