API overview

The public API is at https://api.anpord.com/v1. Every operation is a POST with a JSON body, reads included, and there are no path or query parameters:

POST /v1/evals.batches.start
POST /v1/evals.batches.get
POST /v1/evals.cases.run

Authentication

Pass an API key as a bearer token:

curl https://api.anpord.com/v1/evals.batches.list \
  -H "authorization: Bearer $ANPORD_API_KEY" \
  -H "content-type: application/json" \
  -d '{"limit":20}'

A key belongs to one organization. OAuth access tokens also work. Reading evals needs evals:read. Starting a batch needs evals:write.

Start a batch

evals.batches.start creates a batch that runs every case of a suite on every variant. It returns right away with the batch id and a runs list holding one run per case per variant, each with its caseId and variantId, while the trials continue on the server:

curl https://api.anpord.com/v1/evals.batches.start \
  -H "authorization: Bearer $ANPORD_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "suite": { "id": "smoke", "name": "Smoke", "prompt": "{{task}}" },
    "cases": [{
      "id": "writes-hello",
      "variables": { "task": "Create hello.txt containing exactly hello" },
      "verify": "test \"$(cat hello.txt)\" = hello"
    }],
    "variants": [{
      "harness": "codex",
      "model": "gpt-5.6-sol",
      "sandbox": "daytona"
    }],
    "trials": 3
  }'

trials is the number of trials in each run. verify is the shell command that decides whether a trial passed, which the SDK writes as validate: command("..."). Poll evals.batches.get with the returned id until status is finished or failed. The API can’t run the local sandbox. Run those with anpord eval --local. Starting a batch is not idempotent. If the response is lost, list recent batches with evals.batches.list before retrying.

Run a case again

evals.cases.run starts a batch for a stored case’s newest version. Name variant ids from evals.cases.get to pick variants, or leave variants out to run every variant the case has run on. trials defaults to 1:

curl https://api.anpord.com/v1/evals.cases.run \
  -H "authorization: Bearer $ANPORD_API_KEY" \
  -H "content-type: application/json" \
  -d '{"id":"writes-hello","trials":3}'

Limits

Limit	Value
Trials per run	1 to 10
Cases per batch	100
Variants per batch	20
Trials per batch (cases x variants x trials)	100
Batches running at once per organization	3

A batch over a size limit is refused with 400. A start while three batches are running is refused with 409. Retry once one finishes.

Errors

Status	Meaning
`400`	Invalid request body
`401`	Missing or invalid bearer token
`403`	Missing permission
`404`	Resource not found
`409`	Request conflicts with current state

The errors above return JSON with a message. The TypeScript client decodes responses and throws AnpordError on failure.

Overview

Endpoints

Authentication

Start a batch

Run a case again

Limits

Errors

​Authentication

​Start a batch

​Run a case again

​Limits

​Errors

Authentication

Start a batch

Run a case again

Limits

Errors