github FluidInference/FluidAudio v0.18.4
v0.18.4 — Nemotron streaming on M2/M3 Neural Engines, Paradee v1.3 voices, Cohere ja/zh long-form

one hour ago

Nemotron streaming runs on M2 and M3 Neural Engines

  • English (#1170) and multilingual (#1141, #1116) encoders now ship a v2 bundle (encoder_int8_v2.mlmodelc / encoder_v2.mlmodelc) with the conv-cache slice rewritten, so the M2 and M3 Neural Engines compile it instead of silently running the encoder on the CPU. On an M2 Pro, English 1120 ms goes from 0 to 1,638 ANE ops and from a 311 s to a 12 s first load. Weights and outputs are unchanged (#1171, #1145).
  • StreamingNemotronAsrManager.loadModels(to:) and StreamingNemotronMultilingualAsrManager.downloadVariant fetch it automatically. A cache from an earlier release downloads it once (~565 MB); offline, the old encoder keeps loading. ModelHub.download(repo) still fetches the old English encoder: use loadModels(to:), or copy encoder-v2/<tier>ms/encoder_int8_v2.mlmodelc into the tier's encoder/ folder.
  • Opt-in split 2240 ms multilingual encoder for Neural Engines that reject the single one (#1142).

TTS

  • Paradee-8M v1.3 (#1161): eight voices, adding Hindi (hf_alpha, hm_omega), French (ff_siwis) and Spanish (em_alex), with upstream's eSpeak-free G2P. ParadeeVersion.v13 is the default.
  • Kokoro ANE: opt-in fp32 vocoder, KokoroAneManager(vocoderPrecision: .float32), for robotic onsets on SME chips (A19 Pro, M5). The fp16 vocoder stays the default (#1137, #1138).
  • Pocket TTS: a chunk no longer ends right after a decimal number (#1132).

ASR

  • transcribe(url) and transcribe(samples) resample the same way. The resampler is a setting, ASRConfig.resamplerQuality (default .balanced) (#1134).
  • Long-form v3: a window that ends in a pause keeps its sentence-final punctuation (#1111).
  • Parakeet EOU: rearm(keepTranscript:) rebuilds decoding state after a raw <EOU>, so later words are no longer lost (#1136).
  • Cohere: Japanese and Chinese long-form audio is cut at pauses into chunks of at most 20 s (FLEURS ja CER 20% → 3.8%) (#1146).
  • tdtJa: no repeated tokens at a window seam (#1143).
  • Vocabulary boosting: the rescue similarity floors (0.30 for one word, 0.50 for a span) are on by default, and the original phrase is scored without its punctuation, which removes false swaps (#1152, #1153).

Text normalization: NemoTextProcessing v0.3.4, with normalizeSentence(_:maxSpanTokens:smallNumbersAsWords:). A panic inside the library returns nil instead of aborting the app (#1158).

Reliability

  • A cached compiled bundle missing its program or weights is re-downloaded instead of crashing Core ML on load (#1150).
  • Offline diarizer: embedSpans releases Core ML outputs after each batch; on iOS, memory grew with every meeting (#1148).

What's Changed

  • fix(tts/pocket): don't end a chunk on a decimal number by @unaltolga in #1132
  • fix(audio): give transcribe(url) and transcribe(samples) the same samples (#1112) by @Alex-Wengg in #1134
  • feat(tts/kokoro-ane): opt-in fp32 KokoroVocoder_v2 for robotic onsets on SME chips (#1135) by @Alex-Wengg in #1137
  • feat(tts/kokoro-ane): keep the fp16 vocoder as default; fp32 KokoroVocoder_v2 is opt-in (#1135) by @Alex-Wengg in #1138
  • fix(asr/tdtJa): drop a re-decode's copies of the kept tokens at a window seam (#1139) by @Alex-Wengg in #1143
  • feat(asr/nemotron-multilingual): opt-in split 2240 ms encoder for older Neural Engines (#1116) by @Alex-Wengg in #1142
  • fix(asr/nemotron-multilingual): encoder_v2 so M2/M3 Neural Engines compile the encoder (#1141) by @Alex-Wengg in #1145
  • fix(asr/cohere): cut Japanese and Chinese long-form audio at pauses (#1140) by @Alex-Wengg in #1146
  • feat(asr/eou): add rearm(keepTranscript:) to rebuild decoding state after a raw (#1007) by @amitach in #1136
  • fix(asr): flush pending tokens on an interior window that ends in a pause by @maboa in #1111
  • fix(offline-diarizer): drain Core ML outputs per embedSpans batch by @SeoFood in #1148
  • fix(download): reject compiled bundles missing their program or weights (#1147) by @Alex-Wengg in #1150
  • fix(asr/vocab): rescue similarity floors on by default, score the original as spoken (#1151) by @Alex-Wengg in #1152
  • fix(asr/vocab): punctuation doesn't count toward a word's confidence (#1151) by @Alex-Wengg in #1153
  • ci(ja-benchmark): skip the results comment on fork PRs, fail when posting fails by @Alex-Wengg in #1149
  • feat(itn): bump NemoTextProcessing to v0.3.4, pass smallNumbersAsWords through (#1156) by @Alex-Wengg in #1158
  • feat(tts/paradee): Paradee-8M v1.3 with Hindi, French and Spanish voices and the eSpeak-free G2P (#1159) by @Alex-Wengg in #1161
  • fix(tts): release builds trimmed the first and last word of each phoneme chunk by @Alex-Wengg in #1160
  • fix(asr/nemotron): encoder_int8_v2 so the M2 Neural Engine compiles the English encoder (#1170) by @Alex-Wengg in #1171

Full Changelog: v0.18.3...v0.18.4

Don't miss a new FluidAudio release

NewReleases is sending notifications on new releases.