Chainlit/chainlit 2.0.dev0

on GitHub

Developer Preview

This is a developer preview release of Chainlit 2.0. It introduces significant changes and new features, particularly integration with the OpenAI Realtime API . As a dev preview, it may contain bugs and is not recommended for production use.

Major Changes

Realtime Audio Processing

The most significant change in this release is the introduction of realtime audio processing capabilities, as implemented in PR #1401 by @willydouhard. This feature enables real-time voice conversations with AI assistants.

Check out a screen grab of the demo on Twitter X.

For a practical implementation of this new feature, check out our cookbook entry on creating a realtime assistant.

Breaking Changes in Audio Implementation

• Replaced AudioChunk type with InputAudioChunk and OutputAudioChunk
• Changed default audio sampling rate from 44100 to 24000
• Removed several audio configuration options (min_decibels, initial_silence_timeout, silence_timeout, chunk_duration, max_duration)
• Introduced new on_audio_start callback
• Modified on_audio_end callback to no longer accept file elements as arguments

New Features

• Audio connection signaling with on and off states
• AudioPresence component for visual representation of audio state
• WavRecorder and WavStreamPlayer classes for improved audio handling
• startConversation and endConversation methods in useAudio hook
• Audio interruption functionality

Other Changes

• Updated useChatInteract hook with startAudioStream method
• Modified useChatSession to handle new audio streaming functionality
• Refactored UI components to reflect new audio implementation
• Added new wavtools directory with various audio processing utilities
• Implemented new AudioWorklet processors for more efficient audio handling

Removed

• RecordScreen component
• Several audio-related configuration options from config.toml

For a complete list of changes, please refer to the full changelog.

We encourage developers to test this preview release and provide feedback. Please report any issues or suggestions on our GitHub repository.