github livekit/agents livekit-agents@1.2.0
on GitHub

latest releases: livekit-agents@1.2.8, livekit-agents@1.2.7, livekit-agents@1.2.6...
one month ago

New Features

Evals & Testing:

You can now perform turn-by-turn evaluations on your agent interactions. Here's an example of how to validate expected behaviors: 

result = await sess.run(user_input="Can I book an appointment? What's your availability for the next two weeks?")
result.expect.skip_next_event_if(type="message", role="assistant")
result.expect.next_event().is_function_call(name="list_available_slots")
result.expect.next_event().is_function_call_output()
await result.expect.next_event().is_message(role="assistant").judge(llm, intent="must confirm no availability")

Check out these practical examples: drive-thru, frontdesk

Documentation: https://docs.livekit.io/agents/build/testing/

Preemptive Generation

This feature enables speculative initiation of LLM and TTS processing before the user's turn concludes, significantly reducing response latency by overlapping processing with user audio. Disabled by default: 

session = AgentSession(..., preemptive_generation=True)

Enhanced End-of-Turn (EOU) Detection

The end-of-turn model has been refined to reduce sensitivity to punctuation and better handle multilingual scenarios, notably improving Hindi language support.
'
Documentation: https://docs.livekit.io/agents/build/turns/turn-detector/#supported-languages

OpenTelemetry Integration

Agent now supports tracing for LLM/TTS requests and user callbacks using OpenTelemetry. See LangFuse example for detailed implementation.

Experimental Agent Tasks

AgentTask is a new experimental subset feature allowing agents to terminate upon achieving specific goals. You can await AgentTasks directly in your workflows: 

@function_tool
async def schedule_appointment(self, ctx: RunContext[Userdata], slot_id: str) -> str:
    # Attempts to retrieve user email, allowing multiple agent-user interactions
    email_result = await beta.workflows.GetEmailTask(chat_ctx=self.chat_ctx)

Half-Duplex Pipeline

Combine Gemini or OpenAI's realtime STT/LLM with a separate TTS engine, optimizing your agent's voice interactions: 

session = AgentSession(
    llm=openai.realtime.RealtimeModel(modalities=["text"]),
    # Alternatively: llm=google.beta.realtime.RealtimeModel(modalities=[Modality.TEXT]),
    tts=openai.TTS(voice="ash"),
)

View the complete example.

Documentation: https://docs.livekit.io/agents/integrations/realtime/#separate-tts

Improved Transcription Synchronization

Align transcripts accurately with speech outputs from TTS engines such as Cartesia and 11labs for improved synchronization: 

session = AgentSession(..., use_tts_aligned_transcript=True)

Refer to the complete example.

Documentation: https://docs.livekit.io/agents/build/text/#tts-aligned-transcriptions

Upgraded Tokenization Engine

Transitioned to the Blingfire tokenization engine from the previous naive implementation, significantly enhancing handling and accuracy for multiple languages.

Complete changelog

New Contributors

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.7...livekit-agents@1.2.0

Check out latest releases or
releases around livekit/agents livekit-agents@1.2.0

Don't miss a new agents release

NewReleases is sending notifications on new releases.

Get notifications