Voice

Speech to speech

Running a call speech to speech on one vendor connection instead of the standard recognition, model and synthesis pipeline.

A standard call runs three services in sequence: recognition, model, then synthesis. Choose Call engine on the Build rail to pick between Standard and Speech to speech. While the agent runs on Standard, that row reads Preset. Website chat always uses the reply model.

One connection instead of three

Speech to speech replaces all three with a single connection to a vendor's speech-to-speech model, which listens and speaks directly. You can choose one of three speech-to-speech providers.

Standard           caller audio -> recognition -> model -> synthesis -> caller audio
Speech to speech   caller audio -> vendor speech-to-speech model  -> caller audio

The trade is latency and naturalness against control and resilience. Most settings from the standard pipeline stop applying.

Speech to speech is set on the agent rather than on a version: it applies to every version at once and stays editable on a published agent. While it is on, saving or resetting its settings asks you to confirm, because live calls use the change straight away.

What changes on a call

TTS, STT and LLM stop applying

The voice, recognition engine, recognition language and language model all come from the speech-to-speech vendor instead. Those sections stay in the editor, dimmed, with a note that they don't apply to calls. On an agent that also takes website chat, the language model still answers the chat, except with the full-duplex provider.

The greeting and disclosures are spoken, not played

A standard call plays a pre-rendered opener word for word. Speech to speech has no synthesis lane, so the opener is folded into the model's instructions and the model says it. Recording and AI disclosures fold in the same way, even when the greeting is switched off, because they are compliance copy.

Disclosure wording can vary

Instructed speech is best effort, not deterministic playback, so wording can vary between calls. Calls in this mode are not marked as having played a verbatim disclosure.

There is no failover

A standard call can move between model providers mid-call when one stalls. Speech to speech is one connection: if it drops, the call ends.

No typing overlay

The typing overlay plays through an audio lane this pipeline does not have. The background ambience bed still runs.

Inference happens outside Australia

None of the three speech-to-speech providers has an Australian region. See data residency for how that compares with the standard pipeline.

What stays the same

  • Tools. Every tool is registered the same way, with the same prerequisites, repeat and background handling, and visibility rules.
  • The system prompt, including the security block every agent carries.
  • Transcripts, for both the caller and the agent.
  • Call recordings, including the clean and ambient copies.
  • Caller memory, which still shapes the opener for a recognised caller.

Settings

The provider, model and voice

SettingOptions
Call engineStandard (the default) or Speech to speech. The master switch.
ProviderThe default provider; a second realtime provider with its own models and voices; or a full-duplex provider that listens while it speaks. Each reads its own settings below.
ModelDefault provider: Latest, which tracks its newest release, or one of two pinned generations. Second provider: one of three models. Full-duplex provider: one live model, plus a choice of three backend models that handle reasoning and tool calls.
VoiceDefault provider: one of five built-in voices, or a custom voice ID that overrides them while it is set. Second provider: one of ten voices. Full-duplex provider: one of 22 voices, or a designed voice ID.

Turn handling and the greeting

SettingOptions
Turn handlingThe vendor decides when the caller has finished, or the agent's own turn-taking tuning does. The second provider can also decide on meaning, reading the words to judge whether the caller has finished. The full-duplex provider always decides every turn itself.
Speak greeting firstOn by default. Opens the call with the agent's greeting before continuing.

Choosing the agent's own turn-taking tuning is the one case where a setting outside this page still reaches a realtime call. The turn-taking controls in Transcription stay live and apply.

Search tools are on the default provider only.

SettingOptions
Web searchLets the model search the web mid-call. Off by default.
X searchLets the model search X mid-call, with an optional list of up to 20 allowed handles. Off by default.
Document collectionsCollection IDs held with the vendor, up to 10, separate from the Knowledge base. Any ID switches collection search on.

On this page