Added
-
Added a
custom_configurationfield toNvidiaTTSService.Settings, which forwards model-specific key/value parameters to NVIDIA TTS — Magpie'sflush,chunk_len_thresholdandmax_chunk_thresholdstreaming controls, or Chatterbox'sexaggeration_factor.
(PR #5580) -
Added
ClassifierUserTurnStopStrategyandClassifierUserTurnStrategies, which let a classifier, such asJevClassifier, decide whether the user's turn is complete. They're an alternative toFilterIncompleteUserTurnStrategiesfor an LLM that doesn't follow the turn-completion markers well. At every pause with new words, the classifier judges the turn so far from the user's words and the bot's last line:completeends the turn, andshortandlongkeep it open for a few seconds unless the user continues.ClassifierUserTurnStrategiesadds the classifier to the default stop strategies (smart turn), and the turn ends once both have decided; for the classifier alone, passUserTurnStrategies(stop=[ClassifierUserTurnStopStrategy(...)]).user_params = LLMUserAggregatorParams( user_turn_strategies=ClassifierUserTurnStrategies( classifier=JevClassifier(api_key=os.getenv("TYPESAFE_API_KEY")), context=context, ), )
(PR #5870)
-
Added
number_pronunciation_languagetoSmallestTTSServicesettings, which reads numerals, currency, times and dates in a language of its own, independent of the synthesis language.
(PR #5936) -
Added
text_matchesto scripted eval expectations: one or more regular expressions the event's text must each match, such as a format a reply must open with. A miss fails the expectation with the newtext_unmatchedfailure kind.
(PR #5991) -
LLMWithBackend(predictive_delegation=PredictiveDelegation(classifier))is an optimization that can speed up responses that need backend involvement. It uses a fast yes/no classifier (such asJevClassifier) to determine, after each user request, whether it needs to wait for a frontend inference or can go ahead and trigger an early backend delegation. The cost is money, not time: the classifier runs in parallel with the frontend, but on every turn, and an early delegation abandons a frontend request already under way. Only text frontends are supported, and only request strategies that haveBackendRequestStrategy.delegate_needs_argumentsset toFalse(likeTranscriptBackendRequestStrategy).
(PR #6174) -
Added
AICTytoObserver, which scores the quality of the user's audio with the ai-coustics Tyto model after every 5 seconds of received audio. Each result fires theon_analysis_resultevent with the risk score and per-degradation scores. At the end of the call,on_call_summaryreceives the mean, p95, and max scores, the degraded fraction, and the dominant degradation dimension. Pass the transport'sAICFilterasaudio_filterto score the audio before enhancement.
(PR #6181) -
AWSBedrockLLMServicenow emits the reasoning a model returns inreasoningContent(Claude with extended thinking, Nova 2 with reasoning enabled) asLLMThoughtStartFrame/LLMThoughtTextFrame/LLMThoughtEndFrame. Signed reasoning, such as Claude's, is kept in the context and passed back to the model on later turns, so Claude keeps its reasoning across tool calls.
(PR #6229) -
TavusTransportandTavusVideoServicenow set the Tavus replica's facial emotion from the delivery the bot speaks in, when expressive conversation is enabled (PipelineParams(enable_expressive_conversation=True)). They convert eachTTSTextFrame.deliveryto aTavusEmotionand sendconversation.set_emotionwhen it changes, as the frame's words are heard.TAVUS_DELIVERY_EMOTIONSmaps Pipecat's core deliveries, such aswarmtohappiness, and a universal delivery for each other emotion, such asangrytoanger; any other mood showsneutral. A delivery the expression policy offers that the replica has no emotion for is logged as a warning at startup.
(PR #6291) -
Added the
expressionevent topipecat evalscenarios: the expression a bot's TTS performed, in the tags the LLM writes, such as<<excited>>or[[laughs]]. A tag the TTS can't perform is never reported, so a scenario can check that what the LLM wrote was performed.
(PR #6303) -
Added expressive conversation (alpha), turned on with
PipelineParams(enable_expressive_conversation=True): TTS services perform expression tags the LLM writes into its reply, instead of speaking them. The tag syntax and these APIs are likely to change between releases.<<name>>sets the delivery of the words that follow, such as<<excited>>, until the next delivery tag or the end of the response, and<<neutral>>returns to the voice's baseline.[[sound]]is a sound at one point, such as[[laugh]],[[pause 500]]is a pause in milliseconds, and^^word^^stresses a word.<<slow: A B 7>>speaks just those words in that delivery, on top of the delivery around them.<<slow>>and<<fast>>are deliveries every service that performs them names the same, for saying something carefully, such as a code, or in passing; Cartesia and MiniMax change speed only between sentences.- Each TTS service performs what its expression adapter supports and leaves out the rest, with a warning. Adapters subclass
BaseExpressionAdapterand live inpipecat.adapters.tts, beside the LLM adapters. Adapters ship forCartesiaTTSServiceandCartesiaHttpTTSService,ElevenLabsDialogueTTSService,ElevenLabsHttpTTSServiceon Eleven v3 and v4 models,FishAudioTTSServiceon S2 models, andMiniMaxHttpTTSServiceon speech-2.8 models. A service broadcasts what it can perform at start in aTTSMetadataFrame. - The context records the reply as the LLM wrote it, tags included, and clients see the words without the tags. From a service that reports word timestamps, clients receive each word's
deliveryonbot-output, and each sound and pause where it is performed, as abot-outputsegment of theexpression_soundorexpression_pausetype. A service without word timestamps reports each sentence in the delivery it starts in, after its sounds and pauses. Expression sounds and pauses are sent to clients of RTVI protocol 2.3.0 or later, the protocol version now. - Examples for each provider are in
examples/expressive-conversation/.
(PR #6303)
-
Added the LLM's side of expressive conversation (alpha): with
PipelineParams(enable_expressive_conversation=True), an LLM service teaches itself the expression tags the TTS service it feeds can perform, by their universal names and as a list to keep to, in its system instruction, so a prompt doesn't need to describe them. It reads them from theTTSMetadataFramethat TTS service broadcasts.PipelineParams(expression_policy=...)takes anExpressionPolicy, which every LLM service in the pipeline is taught, each one in anLLMSwitcherincluded. Itsdeliveryandsoundssay which tags are offered, by their universal names, or by default a core set such as<<calm>>,<<warm>>and[[laugh]], and itsguidancelines say when to use them: presets such asExpressionGuidance.MATCH_STYLE, which answers in the user's mood and energy, andExpressionGuidance.SLOW_FOR_DETAILS, which says just a code or a number slowly, in a span such as<<slow: A B 7>>, and the developer's own, joined in order. Tags in the policy the TTS service can't perform are left out, with a warning.- The instruction follows the active TTS service. A
ServiceSwitcherswitch, or a settings change such as a new model that changes what a TTS service performs, broadcasts itsTTSMetadataFrameagain, and the LLM's instruction is rebuilt from it. expression_instructions()returns the instruction for a service's capabilities and a policy, to read or to build a prompt of one's own, andexpression_policy_warning()what that policy asks for that the service can't perform.
(PR #6315)
-
Added
PipecatToneReaderto expressive conversation (alpha), a processor that reads each transcribed user turn with the Pipecat Tone model and sets the reading on itsTranscriptionFrame: the user's likeliest emotions and how likely the turn is to be sarcastic, read from the words against the bot's previous reply. Place it after the STT service; the model downloads from Hugging Face at start and runs on the CPU. Install it withpipecat-ai[tone]. Seeexamples/expressive-conversation/expressive-conversation-tone.py.
(PR #6319) -
Added perception to expressive conversation (alpha): with
PipelineParams(enable_expressive_conversation=True), the LLM reads how the user spoke, as reported by a perception producer such as an STT service or a processor after it.TranscriptionFrame,InterimTranscriptionFrameandEagerTranscriptionFramecarry the reading in a newperceptionfield:PerceptionSignals, each read as the likeliest labels with how sure the reading is, such as a tone offrustrated 0.86, or as a value. An STT service that perceives sets it on the transcriptions it pushes, and a processor after the STT can set it on the transcriptions it passes along. A reading of an interim transcription is provisional. A reading of an eager transcription is written into the user message that an early reply, withenable_eager_end_of_turn, is generated from. The producer declares the signals it reports in aPerceptionMetadataFrameat start.- The user aggregator writes the turn's final reading at the head of its user message, such as
([tone frustrated=0.86, neutral=0.10]) Can you tell me where my order is?, and the LLM service's system instruction explains how to read it: what the confidences mean, that a reading below 0.5 is a guess, and each signal the producer declares.ExpressionPolicyguidance can say what to do with a reading, such as "If the user sounds frustrated at 0.7 or above, apologize once". - Apps get the turn's final reading in
UserTurnStoppedMessage.perception, on the user aggregator'son_user_turn_stoppedevent. Processors after the user aggregator get the reading written into the context inLLMContextUserTurnFrame.perception. Clients get the reading in theperceptionfield of the transcription'suser-inputmessage, which waits for a processor after the STT to read it.
(PR #6319)
-
Added
AssemblyAITTSService, streaming text-to-speech with per-word timings from AssemblyAI's Streaming TTS WebSocket API. Install it withpipecat-ai[assemblyai].pipecat initoffers it as "AssemblyAI".Settings.voiceis sent as the API'svoice_id: any voice by its ID, an AssemblyAI voice by its library slug, such asaai-jane, or a cloned voice by the slug the AssemblyAI dashboard shows. It defaults to Audrey, a voice from AssemblyAI's public voice library.Settings.languageandSettings.modelare optional.- The service connects to the US host; set
urltowss://tts.eu.assemblyai.com/v1/wsto process data in the EU. - Interruptions cancel synthesis in flight without reconnecting. The service keeps an idle session open, replaces a session before its one-hour limit, and waits as long as the server asks before reconnecting.
- See
examples/voice/voice-assemblyai.py.
(PR #6322)
-
Added
UserTurnStrategiesParams(inpipecat.turns.user_turn_strategies) to choose how user turn strategies combine. Pass it asuser_turn_strategies_paramstoLLMUserAggregatorParams,UserTurnProcessororUserTurnController.start_modeandstop_modetake aStrategyMode: withFIRSTthe first strategy to decide wins, and withALLevery strategy has to agree.start_modedefaults toFIRSTandstop_modetoALL.start_merge_modeandstop_merge_modetake aMergeMode:REPLACE, the default, uses the bot's strategies instead of the ones a service recommends, andEXTENDadds them to the service's.# Filter incomplete turns on top of a service's stop strategies. user_params = LLMUserAggregatorParams( user_turn_strategies=FilterIncompleteUserTurnStrategies(), user_turn_strategies_params=UserTurnStrategiesParams( stop_merge_mode=MergeMode.EXTEND, ), )
(PR #6346)
-
Added
BaseUserTurnStopStrategy.decision_scheduled. A stop strategy returns True while it's sure to decide soon on its own, asLLMTurnCompletionUserTurnStopStrategyis while a re-prompt is scheduled, anduser_turn_stop_timeoutthen waits instead of ending the turn.
(PR #6346) -
Added
BaseUserTurnStopStrategy.waits_for_inference. A stop strategy that decides from the turn's inference instead of from the user, such asLLMTurnCompletionUserTurnStopStrategy, returns True. The turn controller runs the inference once the other stop strategies have decided, and that strategy then decides from it.
(PR #6346) -
Added
UserTurnCompletionFrame, which an LLM service using turn-completion markers pushes upstream once per inference. Itskindis"complete","short","long", orNonewhen the response has no marker, and itscontext_frame_idis the id of theLLMContextFramethat started the inference. A tool call before any marker counts as"complete".
(PR #6346) -
Added
ExpressionGuidance.BREATHEto expressive conversation (alpha), a preset for an occasional audible breath before a long or difficult sentence, at most once a reply.PipecatToneTaggerwrites breaths only with it, drawing each as a light breath, a deep breath or an exhale at the shares people take them in, and naming each kind as the TTS service performs it: on ElevenLabs,[quick breath],[deep breath]and[exhales softly].
(PR #6385) -
Added
ExpressionMetadataFrameto expressive conversation (alpha), broadcast at start by a processor between the LLM and the TTS service that writes the expression tags itself. The LLM is then taught no tags, keeping how to read perception and the policy's own guidance lines when there's a reading to act on, and the assistant aggregator writes each reply to the context as its words alone.
(PR #6385) -
Added
PipecatToneTaggerto expressive conversation (alpha), a processor that writes the expression tags for an LLM's plain replies with Pipecat Tone's tagger model. Placed between the LLM and the TTS service, it labels each sentence with a delivery, slow spans for details, stressed words and sounds, from the call so far, how the user sounded and the expression policy's presets, and writes them by their universal names, which the TTS service performs by its own, trying an optionalvocabularyof your own names first. The model downloads from Hugging Face the first time, at an optionalsize("17m"or"32m") andrevision, or comes from an optionalmodel_dirof your own. It loads while the pipeline is set up, once a process for every tagger to share, and runs on the CPU; if it can't load, the LLM writes the tags itself. Install it withpipecat-ai[tone]. Seeexamples/expressive-conversation/expressive-conversation-tone-tagger.py.
(PR #6385) -
Added an optional
revisiontoPipecatToneReader, the model's release on Hugging Face, and an optionalmodel_dirto load the model from a folder of your own instead.
(PR #6407) -
Added an
accentsetting toCartesiaTTSServiceandCartesiaHttpTTSService, which picks the accent a multilingual voice speaks the language with.
(PR #6429) -
Added
moderationtoOpenAIImageGenService.Settings, which sets the content-moderation level OpenAI's Images API applies to generated images ("low"for less restrictive filtering, or"auto").
(PR #6432) -
Added a
pronunciationssetting toSpeechifyHttpTTSService, which tells Speechify how to speak specific words, by IPA phones or by a spoken alias.
(PR #6436) -
Added
TogetherLLMSettings.reasoning_effort, Together's reasoning-depth control ("low", "medium", "high", and "max" on some models). It reaches models that ignore thereasoningtoggle, such aszai-org/GLM-5.3.
(PR #6437) -
Added universal names to expressive conversation (alpha): the
ExpressionDeliveryandExpressionSoundstring enums name each delivery and sound the TTS services with an expression adapter perform, the same on every service. A universal name in a tag, such as<<warm>>or[[laugh]], is sent to each service by its own name, such as ElevenLabs'warmlyor Cartesia'slaughter, while the spoken words'delivery, the sounds reported to clients and the context keep it as written.ExpressionCapabilities.service_vocabularymaps the names that differ, filled in for Cartesia, ElevenLabs, Fish Audio and MiniMax, andExpressionCapabilities.delivery_name()andsound_name()give the name a service performs.
(PR #6439) -
Added expressive conversation (alpha) adapters for five more TTS services, so their LLMs are taught the expression tags they perform. See
examples/expressive-conversation/for each.InworldTTSServiceandInworldHttpTTSServiceoninworld-tts-2perform any delivery as a free-text instruction, such as[calm and slow], with^^emphasis^^in capitals, Inworld's non-verbal sounds and<break>pauses. Oninworld-tts-2-flash, which ignores instructions, only sounds and pauses are performed.SonioxTTSServiceontts-rt-v2performs deliveries and sounds as Soniox's audio tags, such as[excited]and[laughs], with*emphasis*and approximate pauses. Its word timestamps report an audio tag as one word and a stressed word without its asterisks.MistralTTSServiceon Voxtral 3 models (voxtral-mini-tts-3-0) performs a delivery as one of Mistral's emotions, such as[emotion=warm], slow and fast as[pace=slow]and[pace=fast], and sounds such as[laugh]. The default model,voxtral-mini-tts-2603, performs none.GeminiTTSServiceon Gemini 3.8 models with the Gemini API performs any delivery as a style sent with each sentence, added to thepromptsetting's style, with emphasis in capitals and inline sounds and pauses such as<laugh>and<short pause>.AzureTTSServiceandAzureHttpTTSServicewith a DragonHD or DragonHD Omni voice, such asen-US-Ava:DragonHDOmniLatestNeural, perform a delivery as one of Azure's HD styles, such as[excited], and its paralinguistic sounds, such as[laughter].ExpressionCapabilities.allows_free_text_deliveryandallows_free_text_soundssay whether a service takes free text for each kind, as Inworld and Gemini do for delivery alone.ExpressionDeliveryandExpressionSoundgain the universal names these services perform, such asplayful,soft,encouraging,sniffleandwhimper.
(PR #6444)
Changed
-
Updated the
aicextra to requireaic-sdk~=3.3.1(previously~=3.1.0).
(PR #6181) -
LLMWithBackend's frontend acknowledges a delegation to the backend in fewer words, such as "Let me look into that". A longer acknowledgement sounds awkward, and holds up a backend's answer that lands while the acknowledgement is still being spoken.
(PR #6267) -
LLMWithBackend's frontend avoids the awkward double acknowledgement of a delegation. When it has acknowledged in the same response as itsdelegatetool call (for example "Stopping the retry test fix, I'll leave that alone for you."), which it sometimes does, the reply it generates when the call settles completes the thought (for example "Checking CI on main for you now!"), or adds nothing when the first part said it all.
(PR #6314) -
When filtering incomplete turns, a turn the LLM marks ◐ or ○ now stays open until the user is re-prompted (after
UserTurnCompletionConfig.incomplete_short_timeoutorincomplete_long_timeout). Before,user_turn_stop_timeoutended the turn first and firedon_user_turn_stop_timeout.
(PR #6346) -
⚠️ Several stop strategies in one
UserTurnStrategies.stoplist now all have to agree before a user turn ends, becausestop_modedefaults toStrategyMode.ALL. For example,TurnAnalyzerUserTurnStopStrategynext toSpeechTimeoutUserTurnStopStrategynow ends the turn only once both say the user is done. PassUserTurnStrategiesParams(stop_mode=StrategyMode.FIRST)to end the turn on the first stop strategy that decides, as before. A list with a single stop strategy behaves as before.
(PR #6346) -
⚠️
WakePhraseUserTurnStartStrategynow starts turns on its own: once awake, it starts a turn when the user speaks or a final transcription arrives. Use it as the only start strategy,start=[WakePhraseUserTurnStartStrategy(...)]. In aStrategyMode.FIRSTlist, the start strategies after it no longer decide once it's awake. To combine it with others, such asMinWordsUserTurnStartStrategy, usestart_mode=StrategyMode.ALL.
(PR #6346) -
⚠️ A custom stop strategy that decides from the turn's inference, for example one that called
trigger_user_turn_finalized()next todeferred()strategies, must now return True fromBaseUserTurnStopStrategy.waits_for_inference. Otherwise the turn controller treats it as a strategy that listens to the user: in aStrategyMode.ALLlist the inference doesn't run untiluser_turn_stop_timeoutends the turn, and in aStrategyMode.FIRSTlist the turn ends without waiting for it.
(PR #6346) -
FilterIncompleteUserTurnStrategiesnow works with an eager end of turn and with the stop strategies a service recommends (withstop_merge_mode=MergeMode.EXTEND). With an eager end of turn, the speculative answer goes out once the transcript commits and the LLM marks the turn complete, in either order, and ◐ or ○ withdraws it.
(PR #6346) -
⚠️ A user turn stop strategy now only reports that the user is done.
BaseUserTurnStopStrategy.report_user_turn_stopped(), which replacestrigger_user_turn_stopped(), fireson_user_turn_stoppedand no longeron_user_turn_inference_triggered, because the turn controller now runs the turn's inference itself once the stop strategies have decided. A custom stop strategy starts an inference only to speculate, withstart_user_turn_speculation(). The deprecatedtrigger_user_turn_inference_triggered()ignores a call without a speculation, with a warning the first time.
(PR #6346) -
⚠️ Turning on turn-completion markers with only the deprecated
filter_incomplete_user_turnsLLM setting no longer re-prompts a user who goes quiet after ◐ or ○, sinceLLMTurnCompletionUserTurnStopStrategynow does the re-prompting. Useuser_turn_strategies=FilterIncompleteUserTurnStrategies()instead.
(PR #6346) -
When the user speaks again after a turn's answer has started but before the turn ended, that answer is now cancelled: the user aggregator interrupts the bot, or broadcasts a
UserTurnResponseCancelFramewhen the turn doesn't interrupt the bot.UserTurnControllerandUserTurnProcessorhave a newon_user_turn_resumedevent, emitted when the user starts speaking again in an open turn.
(PR #6346) -
ServiceSwitchernow passes aServiceMetadataFramefrom a service outside it to its inactive services too, so a service that reads another service's metadata has it when it becomes active. Only the active service's own metadata leaves the switcher, as before.
(PR #6365) -
With
enable_eager_end_of_turn, the user aggregator also broadcasts anLLMContextUserTurnFramefor the turn an early reply answers, withspeculation=True, ahead of the reply. Processors after the LLM learn the turn before the held reply is released. The turn is broadcast again, as written to the context, when it ends, so a processor that keeps the conversation should skip speculative turns.
(PR #6385) -
Changed the default model of
InceptionLLMServicefrommercury-2tomercury-3. Inception is retiring Mercury 2 and 2.5 on November 15, 2026, and they are closed to new signups. To keep using Mercury 2 until then, setsettings=InceptionLLMService.Settings(model="mercury-2").
(PR #6397) -
SmallestTTSService.Settingsgained adate_formatfield, which reads all-numeric dates day-first ("EU") or month-first ("US").
(PR #6431) -
SmallestTTSService.Settings'content_filternow also takes Smallest's"redact"action, which drops a matched term and speaks the rest.
(PR #6431) -
The MoQ transport example now uses
OpenAILLMService's default model instead of pinninggpt-4o, which OpenAI shuts down on 2027-01-14.
(PR #6433) -
OpenAIResponsesLLMServiceandOpenAIResponsesHttpLLMServicenow sendreasoning.effort="none"forgpt-6.1-terrawhen reasoning isn't configured, matching the other models whose lowest documented effort isnone.
(PR #6434) -
Renamed
RELAY_NOTEtoBACKEND_MESSAGE_RELAY_INSTRUCTIONandSTRATEGY_PLACEHOLDERtoPAIRING_INSTRUCTION_STRATEGY_PLACEHOLDERinpipecat.pipeline.llm_with_backend.
(PR #6435) -
pipecat.pipeline.llm_with_backendnow exports only the classes an app builds with:LLMWithBackend,BackendConnector,PredictiveDelegation, and the request strategies. Import the prompt texts and the marks on backend messages frompipecat.pipeline.llm_with_backend.prompts, andDELEGATE_TOOL_NAMEandConnectorContextfrompipecat.pipeline.llm_with_backend.connector.
(PR #6435) -
Updated
PipecatToneReaderto download the Pipecat Tone reader'sreader-rc2release by default (previouslyreader-rc1), which reads sarcasm with the bot's reply before each turn as context. Passrevision="reader-rc1"to keep rc1.
(PR #6450)
Deprecated
-
Deprecated
BaseUserTurnStartStrategy.trigger_user_turn_started()andBaseUserTurnStopStrategy.trigger_user_turn_stopped(). Usereport_user_turn_started()andreport_user_turn_stopped()instead: a strategy reports its decision, and the turn controller decides when the turn starts and ends. A subclass that overrides the old method keeps getting the reports until 2.0.0.
(PR #6346) -
Deprecated
BaseUserTurnStopStrategy.trigger_user_turn_inference_triggered()andtrigger_user_turn_speculation_cancelled(). Usestart_user_turn_speculation()andcancel_user_turn_speculation()instead.
(PR #6346) -
Deprecated
EagerEndOfTurnCancelFrame. UseUserTurnResponseCancelFrameinstead. Frames pushed under the old name still cancel.
(PR #6346) -
Deprecated
deferred()andDeferredUserTurnStopStrategy, with no replacement. Pass the wrapped strategy itself: the turn controller now runs the turn's inference once the stop strategies have decided.
(PR #6346) -
Deprecated
ProcessFrameResult. UseStrategyModeinstead: user turn strategies no longer need to return anything fromprocess_frame(), and every strategy sees every frame. Until 2.0.0, aStrategyMode.FIRSTlist still honorsProcessFrameResult.STOP, with a deprecation warning.
(PR #6346) -
Deprecated
ExternalUserTurnCompletionStopStrategy. UseExternalUserTurnStopStrategyinstead, and push aProposedUserStoppedSpeakingFrameonce the user is done. For the LLM's turn-completion markers, useLLMTurnCompletionUserTurnStopStrategy.
(PR #6346) -
Deprecated
BaseUserTurnStopStrategy.trigger_user_turn_finalized(). Usereport_user_turn_stopped()instead, which fires the same event.
(PR #6346)
Fixed
-
LLMWithBackend's frontend now more reliably delegates a stop or a change to work it has handed over to the backend. Before, gpt-4.1 and Claude often answered a user's "never mind" themselves, with no call, and the backend carried on with its work. Asked how the work is going before any of the backend's progress has reached it, the frontend now says there is no update yet instead of describing work it cannot see.
(PR #6174) -
NvidiaLLMServicenow recognizes a leading<think>block that follows whitespace, and no longer passes the whitespace between</think>and the answer on to TTS.
(PR #6229) -
AWSBedrockLLMServiceno longer sends Amazon Nova's<thinking>...</thinking>reasoning to TTS. Nova (v1) models write it into the answer text before calling a tool, so voice bots were reading it aloud. A<thinking>block at the start of the response, or of any text the model writes between tool calls, is now emitted asLLMThoughtStartFrame/LLMThoughtTextFrame/LLMThoughtEndFrame.
(PR #6229) -
Fixed TTS services without word timestamps joining streamed sentences in the context with two spaces, and pushing skipped text, such as a code block, ahead of the sentence spoken before it.
(PR #6366) -
Fixed
SkipTagsAggregatorrecognizing only the first of several tag pairs, and never opening a pair whose start and end are the same, such as("**", "**").
(PR #6367) -
Fixed TTS services with word timestamps recording a closing tag written against a word with a space before it in the assistant context, as
<card>4111 </card> is yoursfor<card>4111</card> is yours, and leaving it out when the bot was interrupted right after that word.
(PR #6368) -
Fixed the Pipecat Cloud documentation link printed by
pipecat init(and used in the generated README), which pointed to a page that no longer exists.
(PR #6387) -
Fixed
JevClassifier,ClefClassifierandOpenAIDecisionsClassifierletting a broken TLS connection through as anssl.SSLErrorinstead of aClassifierError, so code that falls back when a classifier can't answer missed it.
(PR #6409) -
Fixed
nanoseconds_to_str()(used when printing a frame'spts) dropping a microsecond through float rounding, e.g. 2.3 s printed as0:00:02.299999.
(PR #6411) -
Fixed
FilterIncompleteUserTurnStrategiesleaving the bot silent when the user made a sound with no words while the LLM was answering, for example right before a tool call. The bot now responds as it does to an empty user turn that interrupted it, followingLLMUserAggregatorParams.empty_user_turn: by default it asks the user to repeat what they said.
(PR #6422) -
Fixed
AudioVolumeTrackerraisingBufferErroronupdatewhen another thread, such as a profiler, held the frame that measured the volume, which stopped VAD for the rest of the session.
(PR #6425) -
Fixed a user turn ending on the user's earlier words with segmented STT services such as
WhisperSTTService,OpenAISTTService,GroqSTTServiceandMoonshineSTTService. When the user spoke again before an earlier segment was transcribed, that segment's transcript was marked finalized, so the stop strategies took it as the user's last words and could end the turn before the new words were transcribed.SegmentedSTTServicenow marks a transcript finalized only when the user hasn't spoken again since its segment ended.
(PR #6446) -
NvidiaTTSServiceno longer freezes the bot while it fetches its synthesis config from the server on startup. A slow NVIDIA endpoint used to block the whole event loop, so nothing else could run and the pipeline's setup timeout couldn't fire until the call returned. The call now runs in a thread.
(PR #6449) -
Fixed segmented STT services (
SegmentedSTTServiceand the services built on it, such asWhisperSTTService,MoonshineSTTServiceandNvidiaSegmentedSTTService) leaving their transcription task running when the pipeline's setup timed out, for example because a service after them couldn't connect. The runner warned about it as a dangling task, and a long-running server never released it.
(PR #6452)
Performance
OpenAIResponsesLLMService(WebSocket) now replies sooner after an interruption: the next request no longer waits for the interrupted response to finish. The two run on separate "lanes" (each with astream_id), so the interrupted one finishes on the side while the next request can still send only the new messages, continuing from the previous response (previous_response_id), rather than the full context.
(PR #6174)