Responses API through LiveKit Inference
You can now use the Responses API with your LiveKit Inference credentials, including structured output and web search. REST is the default; opt into WebSocket transport to reuse connections across turns.
from livekit.agents import inference
llm = inference.responses.LLM("openai/gpt-4.1", use_websocket=True)New providers and models
- Zoom AI: Zoom Scribe STT with live streaming and per-utterance recognition.
- Kalpa:
kalpa-tts-1streaming TTS in English and Hindi. - Rime: STT and Prism speech-to-speech through the standard realtime interface; both require Python 3.11+, and STT requires the
sttextra. - Fish Audio:
transcribe-stream-1realtime STT with word timings and recognition context. - OpenAI:
gpt-realtime-translatethrough the beta translation interface, withgpt-realtime-whisperinput transcription. - Google:
gemini-3.5-live-translate-previewtranslation through Gemini API and Vertex AI. - Respeecher: public Swedish, Brazilian Portuguese, German, and Arabic TTS models.
Security
- Respeecher: moves WebSocket API keys from request URLs to handshake headers, keeping credentials out of URLs.
Behavior changes
- Inference Responses: now defaults to each known model’s lowest reasoning effort; set
reasoning=Noneto retain the provider default. - STT turn detection: late finals after speech resumes join the resumed turn, which waits for STT’s next end-of-speech event.
- Uninterruptible speech: user transcripts are now retained without interrupting playback or generating a reply.
This release also improves realtime history and reconnection, task handoffs, STT timing, transcript synchronization, VAD updates on Python 3.14, worker cleanup, and multichannel TTS fallback.
What's Changed
- fix(realtime): keep item ids within OpenAI's 32-char limit by @russellmartin-livekit in #7958
- docs(mistralai): add missing store option to LLM options example by @detail-app[bot] in #8219
- feat(soniox): pass client_reference_id to the TTS websocket config by @grace0950 in #8236
- fix(thunderphone): use _shorten_id after the openai_utils rename by @chenghao-mou in #8229
- test(drive_thru): replace the eval tests with simulation scenarios graded on the final order by @u9g in #8176
- fix(voice): suspend an AgentTask's awaiter in the run instead of unwatching it by @u9g in #7794
- fix(voice): don't fail the run with a task error its awaiting tool handled by @osekkat in #8269
- Sunset Sarvam TTS model bulbul:v2 and remove bulbul:v3-beta by @dhruvladia-sarvam in #7056
- feat(rime): add Coda custom_lexicon option by @saad-py in #8245
- fix(voice): end a turn on a late STT final after a later END_OF_SPEECH by @acharan-tech-200037 in #7739
- update livekit sdk/api to 1.2/1.2.2 by @davidzhao in #8299
- fix(voice): keep the user turn in the chat context when the current speech cannot be interrupted by @valemicolgarcia in #7922
- chore: remove dead AgentsConsole members from cli.py by @detail-app[bot] in #8277
- fix(worker): close resources when startup fails before run() completes by @BlueX888 in #7942
- fix(deepgram): guard empty languages list in multilingual STT decoding by @Speedyzyzz in #7976
- fix(vad): resize the speech buffer by copying so it works on Python 3.14 by @RitikaxG in #8162
- fix(tools): omit cancel_task from duplicate reject/confirm when not cancellable by @ninglinLiu in #8270
- reduce status update interval to 1.5s by @davidzhao in #8150
- fix(openai): end a failed realtime response without an unretrieved exception by @zek in #8218
- fix(voice): ignore a repeated flush of the same audio segment by @sehaj23 in #8316
- fix(telemetry): keep the stall sampler out of linecache by @zek in #8226
- fix: resolve a failed local turn-detector prediction to the positive default by @chenghao-mou in #8171
- feat(rime): STT Support by @augustohalvarez in #8013
- chore(krisp): bump livekit-plugins-krisp-internal to 0.2.3 by @lukasIO in #8250
- feat(mistralai): expose reasoning effort on the LLM by @muzman123 in #8285
- fix(respeecher): use WebSocket auth headers and add public models by @mitrushchienkova in #8286
- feat(speechmatics): support 8kHz audio and emit_sentences config by @LArmstrongDev in #7952
- chore(deps): update dependency langgraph to v1.2.14 [security] by @renovate[bot] in #8254
- feat: add Zoom AI STT plugin by @cdliao in #7708
- feat(cambai): add speaking_rate option to TTS by @nihaalnz in #8324
- update dependencies by @davidzhao in #8325
- feat(openai): gpt-realtime-translate as a beta TranslationModel by @tinalenguyen in #8105
- fix(stt): anchor transcript timestamps on the session input start by @chenghao-mou in #7795
- fix: report speech end timestamps across stt plugins by @chenghao-mou in #7796
- refactor(openai): Move the Responses engine into livekit-agents by @adrian-cowham in #8188
- test(openai): Add a live probe for multi-message Responses replies by @adrian-cowham in #8209
- feat(inference): Add inference.responses.LLM by @adrian-cowham in #8210
- feat(inference): WebSocket transport for inference.responses.LLM by @adrian-cowham in #8211
- feat(inference): update_options and response_format for inference.responses.LLM by @adrian-cowham in #8212
- docs(examples): Show inference.responses.LLM in basic_agent by @adrian-cowham in #8213
- fix(inference): Don't retry terminal quota denials on inference.responses by @adrian-cowham in #8268
- fix: drop self from the static type of bound function tools by @u9g in #8336
- fix(openai): stop resending the previous output when chaining Responses requests by @longcw in #8085
- fix(google): start a fresh Gemini session on a config change by @longcw in #7944
- fix(voice): don't hide IGNORE_ON_ENTER tools after on_enter returns by @BhaskarKapri07 in #8280
- fix(inference): don't crash on tool_call arguments that arrive before a name by @AKomplished-bug in #8249
- feat(gradium): optional VAD-based start of speech in STT by @davencyw in #8247
- fix(room_io): date a run's end at submission, not after a stalled capture by @u9g in #8168
- fix(aws): hold on to the user-text and deferred-recycle tasks by @Rehansanjay in #7052
- fix(voice): let only the STT end a turn the STT reopened by @chenghao-mou in #8318
- fix(transcription): stop speaking-rate bursts from blocking the event loop by @longcw in #7938
- fix(duplex): keep the words still waiting when the output stream ends by @basnijholt in #8301
- fix(inference): Send agent_context only to STT models that read it by @adrian-cowham in #8365
- feat(google): Gemini 3.5 Live Translate as a beta TranslationModel by @tinalenguyen in #8106
- feat(openai): Add GPT-Live Decisions support by @shawnfeldman in #8338
- feat(fishaudio): add realtime STT by @cshape in #8357
- feat(inference): reasoning= for inference.LLM and inference.responses.LLM by @adrian-cowham in #8358
- feat(inference): default inference.responses.LLM to the model's lowest reasoning effort by @adrian-cowham in #8359
- fix(fireworksai): honor stream language override by @JY-M666 in #8390
- feat(examples): support Tavus alongside LemonSlice in the avatar example by @ggarber in #8131
- feat(voice): send session_closed to the remote session when the session closes by @u9g in #8099
- fix(aws): honor language passed to STT.stream() by @pcbeingused333 in #8374
- fix(tts): preserve channels when resampling fallback audio by @feiiiiii5 in #8393
- feat(rime): add Prism through the standard realtime interface by @naszzz in #7856
- fix(realtime): end a forced ask on its tool call and restore tool_choice once by @longcw in #8395
- feat: report the agent join steps to the server by @boks1971 in #8173
- feat(kalpa): add Kalpa TTS plugin by @hvg639 in #8017
- fix(gpt-live): cap caller audio queued before session.started by @zunkelty in #8387
- fix(ipc): mark supervised processes dead for prometheus by @Ricky-7-Yan in #7017
- feat(inference): Add rime/prism to the realtime models by @tinalenguyen in #8396
- livekit-agents@1.8.11 by @github-actions[bot] in #8397
New Contributors
- @grace0950 made their first contribution in #8236
- @osekkat made their first contribution in #8269
- @saad-py made their first contribution in #8245
- @valemicolgarcia made their first contribution in #7922
- @Speedyzyzz made their first contribution in #7976
- @RitikaxG made their first contribution in #8162
- @zek made their first contribution in #8218
- @sehaj23 made their first contribution in #8316
- @augustohalvarez made their first contribution in #8013
- @muzman123 made their first contribution in #8285
- @LArmstrongDev made their first contribution in #7952
- @cdliao made their first contribution in #7708
- @nihaalnz made their first contribution in #8324
- @AKomplished-bug made their first contribution in #8249
- @davencyw made their first contribution in #8247
- @JY-M666 made their first contribution in #8390
- @hvg639 made their first contribution in #8017
- @zunkelty made their first contribution in #8387
- @Ricky-7-Yan made their first contribution in #7017
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.8.10...livekit-agents@1.8.11