Evaluations
List evaluations
Returns lean evaluation summaries. Fetch a single evaluation to get its aggregates. `q` matches (case-insensitive substring) against an evaluation's `id` or `name`.
Create an autopilot evaluation
The system generates the evaluation spec (tasks/personas/principles) from the grounded corpus, sized server-side by `depth` (no entity counts in the body). One dimension per call — run once per dimension to cover several. Returns the new evaluation + a pollable job.
Batch delete evaluations
Hard-delete up to 100 evaluations by id. Returns lists of deleted and not-found ids. Returns 409 if any matched evaluation is still running.
Create a custom evaluation
References existing task/principle/persona ids — no inline entity creation. Returns the evaluation + a job.
Create a forensic evaluation
Judges caller-provided conversations against the selected principles. A single request body may not exceed 30 MB (a real, enforced limit); there is no cap on the number of conversations.
Delete an evaluation
Delete an evaluation
Get an evaluation
Returns the full evaluation, including its aggregates and the ordered build-progress `steps` — this is the read used for CI gating.
Cancel a running evaluation
Cancel a running evaluation
Rerun an evaluation
Re-executes a prior evaluation. `reuse` replays its exact snapshots; `refresh` re-fetches the current entities (drift). Inherits the source's kind. Forensic evaluations are not rerunnable (409).
Rerun only the failed executions of an evaluation
Re-runs only the failed executions within an existing evaluation, in place — the evaluation re-opens (back to building) and re-finalizes when the reruns complete. Distinct from POST .../rerun, which creates a new evaluation. A no-op (200, reran_count=0, job=null) when there are no failed executions.