Features:
- OpenAI Realtime API now supports video input
What's Changed
- Add Ultravox Realtime API Plugin by @moadel321 in #2992
- fix and clean ultravox realtime API plugin by @longcw in #3339
- remove sentence tokenizer from transcription sync by @longcw in #3334
- non-stream tts: raise no audio pushed error only if text is not empty by @longcw in #3310
- fix speech resume when interrupted by interim transcript by @longcw in #3299
- add preferred_alignment and fix alignment chars by @longcw in #3341
- feat: Add support for Nebius AI Studio Models by @Arindam200 in #3332
- feat(AWS): Enable prompt caching by @itskyf in #3338
- plugins: groq: llm: add support for timeout and max_retries by @ssettle-groq in #3350
- plugins: groq: llm: add support for service_tier by @ssettle-groq in #3348
- Fix warning in model field access by @marctorsoc in #3340
- chore(groq): accept additional options by @davidzhao in #3370
- add ELEVEN_API_KEY to README for configuration by @Panmax in #3371
- feat(azure-stt): add configurable punctuation option to Azure STT by @sarthakgoyal23 in #3326
- fix transcription synchronizer cannot be closed if paused by @longcw in #3378
- chore: add update_options for aws tts by @longcw in #3384
- add text normalization param by @tinalenguyen in #3390
- fix: improve FallbackAdapter streaming capability detection by @bnovik0v in #3294
- fix user speaking span duration by @tinalenguyen in #3404
- add realtime model tool calls to chat ctx by @longcw in #3345
- make flush duration configurable in commit_user_turn by @longcw in #3358
- use markdown and emoji filters for tts_node by default by @longcw in #3305
- commit user turn before closing the AgentSession by @longcw in #3377
- fix OAI realtime response created after generate_reply timeout by @longcw in #3405
- add warning if use_tts_aligned_transcript is enabled but no transcript received from tts by @longcw in #3409
- fix: Agent status was not updated when audio was paused. by @Panmax in #3406
- don't ever send meta to openai by @guidodecaso in #3402
- Revert "tune vad min_silence_duration and min_endpointing_delay (#2953)" by @longcw in #3416
- Removed OpenAI-Beta header as per the docs by @MajorTal in #3412
- add tool call and output to session.history by @longcw in #3316
- fix(sarvam-tts): add bulbul:v3-beta support and make pitch/loudness optional by @21lakshh in #3413
- chore: add keyterms support for AssemblyAI plugin by @dan-ince-aai in #3387
- Revert "Removed OpenAI-Beta header as per the docs" by @longcw in #3419
- feat(Google): Add markup for TTS HD voices by @itskyf in #3281
- feat: update openai realtime API version to GA by @davidzhao in #3420
- revert #3206, do not auto enable use_tts_aligned_transcript by @longcw in #3423
- livekit-agents 1.2.9 by @theomonnom in #3428
New Contributors
- @moadel321 made their first contribution in #2992
- @Arindam200 made their first contribution in #3332
- @ssettle-groq made their first contribution in #3350
- @marctorsoc made their first contribution in #3340
- @sarthakgoyal23 made their first contribution in #3326
- @21lakshh made their first contribution in #3413
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.2.8...livekit-agents@1.2.9