EVALUATIONS

Command	Description
`available-models`	List every modelIdentifier candidate for an evaluation recipe in one call.
`delete`	Delete an evaluation by ID (prompts to confirm unless `--yes`/`--force`).
`download`	Download every evaluation data item to a local JSONL file.
`failure-modes`	List the failure modes for a completed evaluation.
`generate-template`	Print a ready-to-edit evaluation recipe request body.
`get`	Show one evaluation by ID, with scores and per-evaluator detail.
`list`	List evaluations in a project, with optional filter, search, and sort.
`results`	Show per-row evaluation results, one verdict column per evaluator.
`run`	Submit an evaluation from a recipe and wait for it to finish.

AVAILABLE-MODELS

List every modelIdentifier candidate for an evaluation recipe in one call.

oumi-cli evaluations available-models [OPTIONS]

Options

Option	Type	Description
`--project`	str	Project to scope the operation to; falls back to OUMI_PROJECT_ID or the active profile.
`--search`	str	Filter by display name / model name, case-insensitive substring match, applied client-side after merging.
`--limit`	int range	Return at most N rows (default 100), applied to the merged candidate list.
`--all`	flag	Return every candidate (bounded by `--max-items`). Mutually exclusive with `--limit`.
`--max-items`	int range	Hard safety cap on merged rows returned, in any mode (default 10000).

DELETE

Delete an evaluation by ID (prompts to confirm unless --yes/--force).

oumi-cli evaluations delete [OPTIONS] {evaluation_id}

Arguments

Argument	Type	Required	Description
`EVALUATION_ID`	int	✓	ID of the evaluation (see `oumi-cli evaluations list`).

Options

Option	Type	Description
`--project`	str	Project that owns the evaluation; falls back to OUMI_PROJECT_ID or the active profile.
`--force`	flag	Delete without the confirmation prompt.
`--yes`	flag	Delete without the confirmation prompt.

DOWNLOAD

Download every evaluation data item to a local JSONL file.

oumi-cli evaluations download [OPTIONS] {evaluation_id}

Arguments

Argument	Type	Required	Description
`EVALUATION_ID`	int	✓	ID of the evaluation whose data items to download (see `evaluations list`).

Options

Option	Type	Required	Description
`--project`	str		Project that owns the evaluation; falls back to OUMI_PROJECT_ID or the active profile.
`--out`	path		Destination file or directory. Defaults to ./evaluation-<id>-dataitems.jsonl in the current directory.

FAILURE-MODES

List the failure modes for a completed evaluation.

oumi-cli evaluations failure-modes [OPTIONS] {evaluation_id}

Arguments

Argument	Type	Required	Description
`EVALUATION_ID`	int	✓	The ID of the evaluation.

Options

Option	Type	Description
`--project`	str	Project ID. Env: `OUMI_PROJECT_ID`.
`--include-items`	flag	Include per-item failure-mode detail in the response.
`--limit`	int range	Return at most N failure modes (default 100). The CLI fetches and loops over pages internally; you never pass a page token.
`--all`	flag	Return every failure mode, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`.
`--max-items`	int range	Hard safety cap for `--all` (default 10000).

GENERATE-TEMPLATE

Print a ready-to-edit evaluation recipe request body.

oumi-cli evaluations generate-template [OPTIONS]

GET

Show one evaluation by ID, with scores and per-evaluator detail.

oumi-cli evaluations get [OPTIONS] {evaluation_id}

Arguments

Argument	Type	Required	Description
`EVALUATION_ID`	int	✓	ID of the evaluation (see `oumi-cli evaluations list`).

Options

Option	Type	Required	Description
`--project`	str		Project that owns the evaluation; falls back to OUMI_PROJECT_ID or the active profile.

LIST

List evaluations in a project, with optional filter, search, and sort.

oumi-cli evaluations list [OPTIONS]

Options

Option	Type	Description
`--project`	str	Project that owns the evaluation; falls back to OUMI_PROJECT_ID or the active profile.
`--limit`	int range	Return at most N evaluations (default 100). The CLI fetches and loops over pages internally; you never pass a page token.
`--all`	flag	Return every evaluation, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`.
`--max-items`	int range	Absolute cap on rows returned, in every mode (default 10000).
`--search`	str	Keep only evaluations whose display name contains this substring.
`--sort-by`	choice	Field to sort by. One of: `created_at`, `updated_at`.
`--sort-direction`	choice	Sort order (asc is oldest-first).

RESULTS

Show per-row evaluation results, one verdict column per evaluator.

oumi-cli evaluations results [OPTIONS] {evaluation_id}

Arguments

Argument	Type	Required	Description
`EVALUATION_ID`	int	✓	The ID of the evaluation.

Options

Option	Type	Description
`--project`	str	Project ID. Env: `OUMI_PROJECT_ID`.
`--evaluator`	int	Filter rows to a single evaluator’s results (by evaluator ID); applied client-side after fetching.
`--id`	str	Filter to a single data item by its ID; applied client-side after fetching.
`--limit`	int range	Return at most N data items (default 100). The CLI fetches and loops over pages internally; you never pass a page token.
`--all`	flag	Return every data item, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`.
`--max-items`	int range	Hard safety cap for `--all` (default 10000).

RUN

Submit an evaluation from a recipe and wait for it to finish.

oumi-cli evaluations run [OPTIONS]

Options

Option	Type	Required	Description
`--name`	str	✓	Human-readable display name for this evaluation run.
`--recipe`, `-r`	str	✓	Recipe ID (integer) or path to a JSON recipe file (recipeConfig.type must be ‘evaluate’).
`--project`	str		Project to scope the operation to; falls back to OUMI_PROJECT_ID or the active profile.
`--wait` / `--no-wait`	flag		Wait for the run to finish before returning (default); `--no-wait` returns the operation ID immediately.

Getting started

Basics

Workflows

Command reference

EVALUATIONS

AVAILABLE-MODELS

DELETE

DOWNLOAD

FAILURE-MODES

GENERATE-TEMPLATE

GET

LIST

RESULTS

RUN

​AVAILABLE-MODELS

​DELETE

​DOWNLOAD

​FAILURE-MODES

​GENERATE-TEMPLATE

​GET

​LIST

​RESULTS

​RUN

AVAILABLE-MODELS

DELETE

DOWNLOAD

FAILURE-MODES

GENERATE-TEMPLATE

GET

LIST

RESULTS

RUN