ROCm/FastFlowLM v1.0.5

Release v1.0.5

on GitHub

This release adds support for a new model, Hy-MT2-1.8B — a small, quick, multilingual translation model — and fixes a flm bench bug causing degraded performance when benchmarking from long context down to short context.

📦 New Model Support

🌐 Hy-MT2-1.8B

FastFlowLM now supports hy-mt2:1.8b, a small and quick multilingual translation model.

• Tag: hy-mt2:1.8b

Run in CLI mode:

flm run hy-mt2:1.8b

Run in server mode:

flm serve hy-mt2:1.8b

📖 Prompt Guide

Prompt Format

Hy-MT2 is a dedicated translation model, not a general-purpose chat model — it has no default system prompt. There are two ways to prompt it:

Option 1: instruction + text in a single user message

将以下文本翻译为{TARGET_LANG}，注意只需要输出翻译后的结果，不要额外解释：

{TEXT}

or, in English:

Translate the following segment into {TARGET_LANG}, without additional explanation.

{TEXT}

Option 2: instruction as the system prompt, text as the user message

Pin the translation instruction as the system prompt so it isn't re-prefilled every turn, then send only the source text as the user message:

{"role": "system", "content": "将以下文本翻译为英语，注意只需要输出翻译后的结果，不要额外解释。输出必须全部使用英语，不要输出源语言或原文"},
{"role": "user", "content": "{TEXT}"}

This is the recommended format in server mode for multi-turn or repeated translation calls (e.g. batch translating subtitle lines), since the instruction's prefill cost is paid once instead of once per request.

For more details, see the model card and benchmark results.

🐛 Bug Fix: flm bench Performance Degradation Across Context Lengths

Fixed a bug in flm bench where running benchmarks from long context down to short context resulted in progressively poorer performance numbers.

🌟 Summary

Thanks for your support — see you in the next one! 🚀