Test runs
Test runs

Run one test scenario

Run a single test against the agent's current published or draft version. Returns `verdict` (passed / failed / scoring / review_required / error), read as Test analytics reads it: an operator override wins, a passed run whose critical metric failed is failed with `failedCritical` naming the metric, and scoring means the criteria passed but critical metrics are still scoring. Also returns the graded rubric count and (on failure) the judge's reasoning for each failed rubric. A voice test, and over MCP a text test still going after 45 seconds, returns a `verdict` of running with a `batchId` instead: read the verdict with get_test_results, and do not run the test again, which would start a second paid run. After a failure, read the reasoning carefully: the judge often pinpoints exactly which line of the system prompt or which tool description is at fault. Cap at 2 retry passes per test, if the second fix doesn't take, mark it as a known-failing test and move on rather than spinning forever.

POST
/workspaces/{workspace}/agents/{agent}/tests/{test}/runs

Authorization

bearerAuth
AuthorizationBearer <token>

A personal access token from Settings > API tokens.

In: header

Path Parameters

workspace*string

Workspace slug (GET /me lists yours).

agent*string

Agent slug within the workspace.

test*string

Header Parameters

Idempotency-Key?string

A key you choose, such as a UUID, that makes this POST safe to retry. Within 24 hours the same key and request answers with the first successful answer and runs nothing, the same key with a different request answers 422 idempotency_key_reused, and a key whose request is still running answers 409 idempotency_key_in_use. A failed request keeps nothing, so the same key runs it again.

Length1 <= length <= 255

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

application/json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

curl -X POST "https://example.com/workspaces/string/agents/string/tests/string/runs" \  -H "Content-Type: application/json" \  -d '{}'
{
  "verdict": "string",
  "property1": null,
  "property2": null
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}
{
  "type": "http://example.com",
  "title": "string",
  "status": 0,
  "detail": "string",
  "code": "string",
  "errors": [
    {
      "path": "string",
      "message": "string"
    }
  ],
  "estimate": {},
  "retryAfterSeconds": 0
}

Run the test suite POST

Run the agent's test suite as one server-side batch, every eval by default, or a configured sweep: scope with evalIds, repeat each scenario numRuns times (1-50), sweep caller personalities with personaIds (each scenario runs once per persona), cap parallelism with concurrentLimit, and pin a version to evaluate. The batch expands to evalIds × numRuns × personas, so a big sweep can burn significant credits, quote the expanded total to the operator before launching one. Returns immediately with the batch id + total; there is no inline verdict. Call get_test_results with the returned batchId to read per-test outcomes, and cancel_test_batch to stop a runaway batch (queued runs stop; in-flight calls finish). BATCHES SHARE ONE QUEUE at concurrency 3, so a new batch waits behind every outstanding run of this agent's earlier ones: the response reports `runsAhead` (and `batchesAhead`) so you can see the wait before you start polling. When you are re-running after an edit and the earlier batch is testing the version you just replaced, pass supersede: true, or the new runs sit behind work you have already invalidated. Supersede REQUESTS cancellation rather than stopping the earlier batches on the spot: their in-flight runs finish, their queued runs never start, and the queue slots free up as those runs drain, so `runsAhead` can still be non-zero for a minute or two after a supersede, and the superseded batches (returned as `supersededBatches`) read as 'cancelling' until their last run lands.

Get a call: transcript, metrics, tool calls and analysis GET

What happened on one call of this agent, in one read. By default returns the compact view: the session row (end reason, the carrier's call control and leg ids, hangup cause, carrier quality); the agent and the version the call ran on, as identity only (version id and number, model, voice, tool names); the full transcript with untruncated tool calls and results and interruption times; per-turn latency (every voice_turn_timing measurement that has a value); pipeline spans; tool call events; audio quality; the analysis summary; every metric result with its key, name, verdict and reasoning; turn-taking and carrier events; message deliveries, post-call actions and outcomes; signed links to the clean, ambient and carrier recordings (valid 24 hours; stereo, caller on the left channel, agent on the right), when the workspace's recording download setting admits your role (otherwise `recordings.linksWithheld` says so and each recording carries its path only); and, for platform staff only, links to the call's trace, logs and error reports, and the runtime's stream and machine ids. `unavailable` names any source that could not be read. The transcript, tool call events, function calls, detail captures and word timings arrive as JSON inside an untrusted-data block. Rows leave out their ids, the columns that repeat the call and the columns that are null. `omitted` names each part and column the compact view leaves out. Pass `section` to read one part without the rest of the call: `agentConfig` (the agent's settings, and the version's system prompt, settings and variables), `tools` (the version's tool definitions with descriptions, params and requires), `metricDefinitions` (the agent's active metric definitions), `metricDetail` (every metric result with its detail payload), `telnyxPayloads` (every carrier event with its raw payload) or `wordTimings` (the start and end of each word the caller spoke, keyed by message id and sequence). `section: "full"` returns the whole bundle in one reply, several times what a client shows inline.