Call checks
How every call is scored, what it costs, and where the scores appear.
Call checks decide what counts as a good call. The same checks that grade an evaluation also grade real customer calls, so a failing check on Monday's calls is directly comparable to a failing check in a test.
Every check on the Call checks page is on unless you switch it off. A new agent is fully scored from its first call, with no setup.
What runs after a call
When a call ends, three passes run.
- Deterministic checks. Measured from the transcript, the timings and the recording. No language model is involved, so they cost nothing and always run.
- Judge metrics. A language model reads the transcript and answers one question per metric. Only the metrics active on this agent are judged, so an agent with no active judge metrics makes no model calls and is charged nothing for this pass.
- Post-call analysis. One model pass that writes the summary, sentiment,
outcome and fields, and rolls the call into
caller memory when caller memory is on. Post-call
functions, follow-ups, alerts and the
call_log.completedwebhook run after it.
Post-call analysis has its own switch, Post-call analysis, in the
Post-call analysis panel on the agent's Automations page. It covers every version.
Turning it off stops the summary, outcome, fields, caller memory updates,
post-call functions, follow-ups, alerts and the call_log.completed webhook.
Checks still run and are still charged.
Every evaluation, text, bridge or phone, runs the same chain marked as simulated. The checks, the analysis and the post-call functions run, but a post-call function that would send something records what it would have sent instead, and no webhook is sent.
The call checks page
Call checks is under the agent's Settings. A change applies to every version straight away. The page has three parts:
- Grading model. The model that grades the judge metrics, in tests and on real calls. Pick one, or choose Use the default for the workspace's.
- Checks. The checks you can switch off, grouped under Conversation quality, Caller experience, and Resolution and reliability. Each shows its price per call and a note explaining when switching it off makes sense.
- Always on. Paid checks with no switch, because a failure means something went wrong rather than that this agent works differently. The card's note gives their combined price per call.
Free checks are not listed, which is why switching everything visible off does not switch scoring off. A check that does not apply to a call sits it out rather than failing it.
How a call is judged successful
A call is successful when its outcome is Completed, or Transferred and the transfer is not known to have failed. Post-call analysis sets the outcome. Checks are shown for review and do not change a call's outcome.
In an evaluation, a failed critical check still fails the run. A check is critical if its definition says it affects call success. Four do today: task success, hallucination, tool call success and disclosure spoken.
What scoring costs
Call checks are billed per active judge metric on the call, never as a flat rate. On a per-minute plan rate or a custom rate, checks are included in that rate, and the Checks card says so.
- Deterministic checks are free.
- Each judge metric has its own price, shown on the Call checks page next to its switch.
- Only a check that actually produced a result is billed. Errored, untriggered or skipped checks are not.
See Call check catalogue for the full catalogue and which single deterministic check is not free.
Test runs are charged differently. Grading in a test run is charged by model usage. The run estimate before launching an evaluation adds up call time and the checks each voice scenario is graded on, plus the agent's replies and the grading. Treat any estimate as an estimate: billing follows the metered minutes and what actually ran.
Switching off a judge metric that does not suit the line removes its price from every call. That is what the Checks card is for.
Where scores appear
Scores land on the call, not in a separate report.
The call page opens with a summary card, a result card and an analysis card. The analysis card shows what post-call analysis recorded: call successful, urgency, caller type and your own fields. The result card carries a headline strip (task success, hallucination, CSAT and latency), a count of checks passed, failed and measured, and the failing checks, each linking to its row. Metrics that errored or were skipped are left out of the strip.
The Checks tab lists every check as a row with its verdict or the value it measured. Opening a row shows the judge's reasoning, the judge prompt and a thumbs-down to flag a verdict you disagree with. Run checks scores a call that has none, and Run again scores it again.
Rated checks
A rated check has no pass or fail. It is a score out of 100, shown as a percent.
CSAT is built from three ratings the judge gives the call, each from 0 to 4:
- Resolution, 45% of the score: whether the caller got what they called for.
- Ease, 30%: how little the caller had to repeat or correct.
- Mood, 25%: how the caller sounded at the end of the call.
Opening the CSAT row shows all three.
On the calls list, the Checks column shows how many pass-or-fail checks failed, or that they all passed, and its tooltip names up to five failing checks.
Related
Call handling
The call-lifecycle limits on an agent - maximum call duration, silent callers, hanging up on an abusive caller, and what a blocked caller hears.
Call check catalogue
The two kinds of check, the catalogue the platform runs on a call, which checks you can switch off, and the shapes a metric result takes.