ROCm/FastFlowLM v1.0.8

Release v1.0.8

on GitHub

This release adds minicpm-v-4.7:1b, a 1B vision-language model with toggleable thinking and tool calling. It also opens up the source of the Gemma 4 engine, so you can build it yourself and swap individual operators through new override hooks. And for the first time, the release ships HRX packages as a preview: builds of FastFlowLM that run on the HRX NPU runtime instead of XRT.

📦 New Model Support

👁️ MiniCPM-V-4.7-1B

FastFlowLM now supports minicpm-v-4.7:1b, a compact image-text-to-text model from OpenBMB, with toggleable thinking and tool calling.

• Tag: minicpm-v-4.7:1b
• Context: 32k by default, up to 256k

Run in CLI mode:

flm run minicpm-v-4.7:1b

Run in server mode:

flm serve minicpm-v-4.7:1b

🖼️ How It Sees Images

MiniCPM-V-4.7 handles images the same way the original MiniCPM-V does:

• The model always sees a downscaled view of the whole image. Large images are also split into up to 9 tiles of about 448 × 448 pixels each.
• Image TTFT stops growing at about 1080p. From that size up, every image uses the maximum 9 tiles, so a 4K image costs about the same as a 1080p one. Very fine detail in large images is downscaled.
• --img-pre-resize (-r) has no effect on this model, because the tiling already decides the input resolution.
• Video and audio input are not supported.

🎛️ MiniCPM-V does not ship its own sampling defaults. FLM starts from temperature 0.7, top_p 0.8, top_k 20 and presence penalty 1.5. These are the Qwen3.5 non-thinking defaults, because MiniCPM-V-4.7-1B's language model has the same architecture as Qwen3.5-0.8B.

For more details, see the model card and benchmark results.

🔓 The Gemma 4 Engine Is Now Open Source

PR #763

Until now, every model engine in FastFlowLM shipped only as a prebuilt library. In this release the source of the Gemma 4 E2B/E4B engine is public. It covers the text decode and prefill paths, the vision encoder and the audio encoder, plus the shared host code they depend on. You can now read exactly how each Gemma 4 operator is dispatched to the NPU, and build the engine yourself. The other engines, including the Gemma 4 flash and 12B variants, still ship prebuilt.

🛠️ Build It from Source

Source builds are opt-in. By default, FLM still links the prebuilt engine, so nothing changes unless you ask for it.

Linux:

cd src
cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON
cmake --build build

Windows (developer command prompt):

cd src
cmake --preset windows-default -DFLM_BUILD_GEMMA4E=ON
cmake --build build

The source-built library goes into the build tree and does not overwrite the prebuilt one. CMake caches the option, so to switch back, reconfigure with -DFLM_BUILD_GEMMA4E=OFF or use a fresh build directory.

Source builds work on Linux, Windows and the HRX backend. CI now builds and sweep-tests a source-built Gemma 4 package on both Linux and Windows.

🧩 Operator Override Hooks for IRON

Every Gemma 4 operator dispatch (52 sites across text, vision, audio and weight preload) is now wrapped in an FLM_OVERRIDE annotation. In a normal build the annotation expands to the original call, so the generated code is identical. To change that, point FLM_OVERRIDES at a header of your own, and that header can replace any operator by name. For example, it can run a kernel written with IRON in place of the stock one. Pass the define through FLM_OVERRIDE_FLAGS.

This lets you try a new NPU kernel inside a full Gemma 4 model without forking the engine. See src/include/flm_override.hpp for the details.

For the full build instructions, see the Linux install guide.

🧪 HRX Packages (Preview)

FastFlowLM can run on two NPU runtimes: XRT (the default) and HRX, the amdxdna runtime. Starting with this release, HRX builds are published as a preview, next to the standard ones:

Each HRX package bundles the HRX runtime and HRX builds of the model engines. The HRX packages cover the same model lineup as the standard ones, including MiniCPM-V-4.7, Gemma 4 and Qwen3.8-27B. They are tested on every pull request by the same model sweep as the standard packages.

Installing works exactly like the standard portable packages: download the HRX asset for your platform and install it the same way you would install fastflowlm_1.0.8_linux.tar.gz or fastflowlm_1.0.8_windows_amd64.zip.

⚠️ HRX is a preview. Expect rough edges, and please open an issue if something works on the standard package but not on the HRX one. The standard XRT packages and installers are unchanged; if you are not sure which one you need, keep using the standard package.

🌟 Summary

Thanks for your support — see you in the next one! 🚀