ROCm/FastFlowLM v0.9.3

🖼️ FastFlowLM v0.9.3 — Vision with Gemma3, Quant Updates & Template Fixes

on GitHub

FastFlowLM v0.9.3 introduces vision support for Gemma3, refined quantization for Qwen3, and multiple bug fixes for better context and prompt handling.

✨ What's New

🧠 1. Vision Support for gemma3:4b

• Now supports vision-language inference with gemma3:4b (VLM).
• Use images in both CLI and REST API:
  • CLI:

    /input "path/to/image.png" What’s in this image?
    

    • Supports .png and .jpg formats.
  • REST API (/api/chat): Provide images using the "images" field.
    • Supports .png and .jpg formats.
• Original text-only model is renamed to: gemma3-text:4b

  Use the text-only version for lower memory usage on pure text tasks.

📉 2. Quantization Update for Qwen3

• Updated qwen3:4b and qwen3:8b to use Unsloth GGUF quantization.
• Provides better performance and compatibility.

🐞 Bug Fixes

• 🧼 Context Preservation in Gemma3 CLI:
  Fixed an issue where chat history was cleared after manually increasing the context length.

• ⚙️ Qwen3 Chat Template Fix:
  Proper support for user-defined system prompts in CLI mode is now enabled.

✅ Usage Recap

Run Gemma3 VLM (Image + Text):

/input "path/to/image.png" What’s happening here?

Run Text-Only Gemma3:

flm run gemma3-text:4b

This release brings multi-modal capability to FastFlowLM while enhancing performance and flexibility across models.