Nemotron streaming runs on M2 and M3 Neural Engines
- English (#1170) and multilingual (#1141, #1116) encoders now ship a v2 bundle (
encoder_int8_v2.mlmodelc/encoder_v2.mlmodelc) with the conv-cache slice rewritten, so the M2 and M3 Neural Engines compile it instead of silently running the encoder on the CPU. On an M2 Pro, English 1120 ms goes from 0 to 1,638 ANE ops and from a 311 s to a 12 s first load. Weights and outputs are unchanged (#1171, #1145). StreamingNemotronAsrManager.loadModels(to:)andStreamingNemotronMultilingualAsrManager.downloadVariantfetch it automatically. A cache from an earlier release downloads it once (~565 MB); offline, the old encoder keeps loading.ModelHub.download(repo)still fetches the old English encoder: useloadModels(to:), or copyencoder-v2/<tier>ms/encoder_int8_v2.mlmodelcinto the tier'sencoder/folder.- Opt-in split 2240 ms multilingual encoder for Neural Engines that reject the single one (#1142).
TTS
- Paradee-8M v1.3 (#1161): eight voices, adding Hindi (
hf_alpha,hm_omega), French (ff_siwis) and Spanish (em_alex), with upstream's eSpeak-free G2P.ParadeeVersion.v13is the default. - Kokoro ANE: opt-in fp32 vocoder,
KokoroAneManager(vocoderPrecision: .float32), for robotic onsets on SME chips (A19 Pro, M5). The fp16 vocoder stays the default (#1137, #1138). - Pocket TTS: a chunk no longer ends right after a decimal number (#1132).
ASR
transcribe(url)andtranscribe(samples)resample the same way. The resampler is a setting,ASRConfig.resamplerQuality(default.balanced) (#1134).- Long-form v3: a window that ends in a pause keeps its sentence-final punctuation (#1111).
- Parakeet EOU:
rearm(keepTranscript:)rebuilds decoding state after a raw<EOU>, so later words are no longer lost (#1136). - Cohere: Japanese and Chinese long-form audio is cut at pauses into chunks of at most 20 s (FLEURS ja CER 20% → 3.8%) (#1146).
- tdtJa: no repeated tokens at a window seam (#1143).
- Vocabulary boosting: the rescue similarity floors (0.30 for one word, 0.50 for a span) are on by default, and the original phrase is scored without its punctuation, which removes false swaps (#1152, #1153).
Text normalization: NemoTextProcessing v0.3.4, with normalizeSentence(_:maxSpanTokens:smallNumbersAsWords:). A panic inside the library returns nil instead of aborting the app (#1158).
Reliability
- A cached compiled bundle missing its program or weights is re-downloaded instead of crashing Core ML on load (#1150).
- Offline diarizer:
embedSpansreleases Core ML outputs after each batch; on iOS, memory grew with every meeting (#1148).
What's Changed
- fix(tts/pocket): don't end a chunk on a decimal number by @unaltolga in #1132
- fix(audio): give transcribe(url) and transcribe(samples) the same samples (#1112) by @Alex-Wengg in #1134
- feat(tts/kokoro-ane): opt-in fp32 KokoroVocoder_v2 for robotic onsets on SME chips (#1135) by @Alex-Wengg in #1137
- feat(tts/kokoro-ane): keep the fp16 vocoder as default; fp32 KokoroVocoder_v2 is opt-in (#1135) by @Alex-Wengg in #1138
- fix(asr/tdtJa): drop a re-decode's copies of the kept tokens at a window seam (#1139) by @Alex-Wengg in #1143
- feat(asr/nemotron-multilingual): opt-in split 2240 ms encoder for older Neural Engines (#1116) by @Alex-Wengg in #1142
- fix(asr/nemotron-multilingual): encoder_v2 so M2/M3 Neural Engines compile the encoder (#1141) by @Alex-Wengg in #1145
- fix(asr/cohere): cut Japanese and Chinese long-form audio at pauses (#1140) by @Alex-Wengg in #1146
- feat(asr/eou): add rearm(keepTranscript:) to rebuild decoding state after a raw (#1007) by @amitach in #1136
- fix(asr): flush pending tokens on an interior window that ends in a pause by @maboa in #1111
- fix(offline-diarizer): drain Core ML outputs per embedSpans batch by @SeoFood in #1148
- fix(download): reject compiled bundles missing their program or weights (#1147) by @Alex-Wengg in #1150
- fix(asr/vocab): rescue similarity floors on by default, score the original as spoken (#1151) by @Alex-Wengg in #1152
- fix(asr/vocab): punctuation doesn't count toward a word's confidence (#1151) by @Alex-Wengg in #1153
- ci(ja-benchmark): skip the results comment on fork PRs, fail when posting fails by @Alex-Wengg in #1149
- feat(itn): bump NemoTextProcessing to v0.3.4, pass smallNumbersAsWords through (#1156) by @Alex-Wengg in #1158
- feat(tts/paradee): Paradee-8M v1.3 with Hindi, French and Spanish voices and the eSpeak-free G2P (#1159) by @Alex-Wengg in #1161
- fix(tts): release builds trimmed the first and last word of each phoneme chunk by @Alex-Wengg in #1160
- fix(asr/nemotron): encoder_int8_v2 so the M2 Neural Engine compiles the English encoder (#1170) by @Alex-Wengg in #1171
Full Changelog: v0.18.3...v0.18.4