Create test set from run
Freeze ONE finished eval run's metric corpus into a named, re-scoreable regression set, promoting the judge's verdict at the time the run finished into ground truth (NOT operator votes; create_metric_test_set does that). Use to lock in a run's judged results before changing a metric prompt, then score_metric_test_set to check the new judge still agrees. Fails when the run was never evaluated for the metric. Names are unique per workspace. Returns the set id + member count.
Authorization
bearerAuth A personal access token from Settings > API tokens.
In: header
Header Parameters
A key you choose, such as a UUID, that makes this POST safe to retry. Within 24 hours the same key and request answers with the first successful answer and runs nothing, the same key with a different request answers 422 idempotency_key_reused, and a key whose request is still running answers 409 idempotency_key_in_use. A failed request keeps nothing, so the same key runs it again.
1 <= length <= 255Request Body
application/json
TypeScript Definitions
Use the request body type in TypeScript.
Response Body
application/json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
curl -X POST "https://example.com/operations/create_from_run" \ -H "Content-Type: application/json" \ -d '{ "name": "string", "runId": "string", "metricKey": "string", "agent": "string" }'{}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}{
"type": "http://example.com",
"title": "string",
"status": 0,
"detail": "string",
"code": "string",
"errors": [
{
"path": "string",
"message": "string"
}
],
"estimate": {},
"retryAfterSeconds": 0
}Create test set from call log POST
Freeze ONE call log's metric verdict into a named, re-scoreable regression set, promoting the judge's verdict for that call into ground truth (NOT operator votes). The call-scoped twin of create_from_run. Use to lock in a notable call's judged result before changing a metric prompt, then score_metric_test_set to check the new judge still agrees. Fails when the call was never evaluated for the metric (run evaluate_call first). Names are unique per workspace. Returns the set id + member count.
Create persona POST
Create a custom caller persona in the workspace catalogue. A persona is the SIMULATED CALLER a test scenario runs as, its `prompt` is the behaviour brief (who they are, what they want, how they speak). Use this to add a caller type the predefined catalogue doesn't cover (e.g. an angry customer, a silent caller, someone with a thick accent). The persona is workspace-scoped (reusable across every agent's tests) and is NOT tied to any one scenario. Slugs are unique per workspace; a duplicate is rejected. Call list_personas first so you don't duplicate an existing persona.