Speech to speech
Running a call speech to speech on one vendor connection instead of the standard recognition, model and synthesis pipeline.
A standard call runs three services in sequence: recognition, model, then synthesis. Choose Call engine on the Build rail to pick between Standard and Speech to speech. While the agent runs on Standard, that row reads Preset. Website chat always uses the reply model.
One connection instead of three
Speech to speech replaces all three with a single connection to a vendor's speech-to-speech model, which listens and speaks directly. You can choose one of three speech-to-speech providers.
Standard caller audio -> recognition -> model -> synthesis -> caller audio
Speech to speech caller audio -> vendor speech-to-speech model -> caller audioThe trade is latency and naturalness against control and resilience. Most settings from the standard pipeline stop applying.
Speech to speech is set on the agent rather than on a version: it applies to every version at once and stays editable on a published agent. While it is on, saving or resetting its settings asks you to confirm, because live calls use the change straight away.
What changes on a call
TTS, STT and LLM stop applying
The voice, recognition engine, recognition language and language model all come from the speech-to-speech vendor instead. Those sections stay in the editor, dimmed, with a note that they don't apply to calls. On an agent that also takes website chat, the language model still answers the chat, except with the full-duplex provider.
The greeting and disclosures are spoken, not played
A standard call plays a pre-rendered opener word for word. Speech to speech has no synthesis lane, so the opener is folded into the model's instructions and the model says it. Recording and AI disclosures fold in the same way, even when the greeting is switched off, because they are compliance copy.
Disclosure wording can vary
Instructed speech is best effort, not deterministic playback, so wording can vary between calls. Calls in this mode are not marked as having played a verbatim disclosure.
There is no failover
A standard call can move between model providers mid-call when one stalls. Speech to speech is one connection: if it drops, the call ends.
No typing overlay
The typing overlay plays through an audio lane this pipeline does not have. The background ambience bed still runs.
Inference happens outside Australia
None of the three speech-to-speech providers has an Australian region. See data residency for how that compares with the standard pipeline.
What stays the same
- Tools. Every tool is registered the same way, with the same prerequisites, repeat and background handling, and visibility rules.
- The system prompt, including the security block every agent carries.
- Transcripts, for both the caller and the agent.
- Call recordings, including the clean and ambient copies.
- Caller memory, which still shapes the opener for a recognised caller.
Settings
The provider, model and voice
| Setting | Options |
|---|---|
| Call engine | Standard (the default) or Speech to speech. The master switch. |
| Provider | The default provider; a second realtime provider with its own models and voices; or a full-duplex provider that listens while it speaks. Each reads its own settings below. |
| Model | Default provider: Latest, which tracks its newest release, or one of two pinned generations. Second provider: one of three models. Full-duplex provider: one live model, plus a choice of three backend models that handle reasoning and tool calls. |
| Voice | Default provider: one of five built-in voices, or a custom voice ID that overrides them while it is set. Second provider: one of ten voices. Full-duplex provider: one of 22 voices, or a designed voice ID. |
Turn handling and the greeting
| Setting | Options |
|---|---|
| Turn handling | The vendor decides when the caller has finished, or the agent's own turn-taking tuning does. The second provider can also decide on meaning, reading the words to judge whether the caller has finished. The full-duplex provider always decides every turn itself. |
| Speak greeting first | On by default. Opens the call with the agent's greeting before continuing. |
Choosing the agent's own turn-taking tuning is the one case where a setting outside this page still reaches a realtime call. The turn-taking controls in Transcription stay live and apply.
Search
Search tools are on the default provider only.
| Setting | Options |
|---|---|
| Web search | Lets the model search the web mid-call. Off by default. |
| X search | Lets the model search X mid-call, with an optional list of up to 20 allowed handles. Off by default. |
| Document collections | Collection IDs held with the vendor, up to 10, separate from the Knowledge base. Any ID switches collection search on. |
Related
Background sound
The background sound bed on a call, the tool sound that covers a lookup, and named pauses.
Answering hours and the closed line
What happens to a call outside your schedule. By default it never reaches a language model, and the runtime plays your closed line in the agent's own voice, then hangs up.