Skip to main content
Each entry in tasks defines one variant:
Today, harness must be codex; Claude Code is not supported yet. The field stays explicit so future harnesses can use the same case format.

Choose a model

Keep the model id in application or CI configuration. Use the catalogue when choosing or changing it:
Model discovery is not required before evals.start. Available ids depend on the account configured on the server.

Choose a sandbox

Each trial gets a separate workspace. Setup, agent work, and verification run in that workspace.
local is not isolated from the server process. The public API and SDK do not accept it.

Compare variants

Change one field at a time when you want to explain the difference:
Changing the harness, harness version, model, or provider creates a new cell history and baseline.