> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# EVALUATIONS

> Run evaluations and inspect how a model scored

| Command                                   | Description                                                                |
| ----------------------------------------- | -------------------------------------------------------------------------- |
| [`available-models`](#available-models)   | List every modelIdentifier candidate for an evaluation recipe in one call. |
| [`delete`](#delete)                       | Delete an evaluation by ID (prompts to confirm unless `--yes`/`--force`).  |
| [`download`](#download)                   | Download every evaluation data item to a local JSONL file.                 |
| [`failure-modes`](#failure-modes)         | List the failure modes for a completed evaluation.                         |
| [`generate-template`](#generate-template) | Print a ready-to-edit evaluation recipe request body.                      |
| [`get`](#get)                             | Show one evaluation by ID, with scores and per-evaluator detail.           |
| [`list`](#list)                           | List evaluations in a project, with optional filter, search, and sort.     |
| [`results`](#results)                     | Show per-row evaluation results, one verdict column per evaluator.         |
| [`run`](#run)                             | Submit an evaluation from a recipe and wait for it to finish.              |

***

## AVAILABLE-MODELS

List every modelIdentifier candidate for an evaluation recipe in one call.

```bash theme={null}
oumi-cli evaluations available-models [OPTIONS]
```

**Options**

| Option        | Type      | Required | Description                                                                                               |
| ------------- | --------- | -------- | --------------------------------------------------------------------------------------------------------- |
| `--project`   | str       |          | Project to scope the operation to; falls back to OUMI\_PROJECT\_ID or the active profile.                 |
| `--search`    | str       |          | Filter by display name / model name, case-insensitive substring match, applied client-side after merging. |
| `--limit`     | int range |          | Return at most N rows (default 100), applied to the merged candidate list.                                |
| `--all`       | flag      |          | Return every candidate (bounded by `--max-items`). Mutually exclusive with `--limit`.                     |
| `--max-items` | int range |          | Hard safety cap on merged rows returned, in any mode (default 10000).                                     |

***

## DELETE

Delete an evaluation by ID (prompts to confirm unless `--yes`/`--force`).

```bash theme={null}
oumi-cli evaluations delete [OPTIONS] {evaluation_id}
```

**Arguments**

| Argument        | Type | Required | Description                                             |
| --------------- | ---- | -------- | ------------------------------------------------------- |
| `EVALUATION_ID` | int  | ✓        | ID of the evaluation (see `oumi-cli evaluations list`). |

**Options**

| Option      | Type | Required | Description                                                                              |
| ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- |
| `--project` | str  |          | Project that owns the evaluation; falls back to OUMI\_PROJECT\_ID or the active profile. |
| `--force`   | flag |          | Delete without the confirmation prompt.                                                  |
| `--yes`     | flag |          | Delete without the confirmation prompt.                                                  |

***

## DOWNLOAD

Download every evaluation data item to a local JSONL file.

```bash theme={null}
oumi-cli evaluations download [OPTIONS] {evaluation_id}
```

**Arguments**

| Argument        | Type | Required | Description                                                                 |
| --------------- | ---- | -------- | --------------------------------------------------------------------------- |
| `EVALUATION_ID` | int  | ✓        | ID of the evaluation whose data items to download (see `evaluations list`). |

**Options**

| Option      | Type | Required | Description                                                                                             |
| ----------- | ---- | -------- | ------------------------------------------------------------------------------------------------------- |
| `--project` | str  |          | Project that owns the evaluation; falls back to OUMI\_PROJECT\_ID or the active profile.                |
| `--out`     | path |          | Destination file or directory. Defaults to ./evaluation-\<id>-dataitems.jsonl in the current directory. |

***

## FAILURE-MODES

List the failure modes for a completed evaluation.

```bash theme={null}
oumi-cli evaluations failure-modes [OPTIONS] {evaluation_id}
```

**Arguments**

| Argument        | Type | Required | Description               |
| --------------- | ---- | -------- | ------------------------- |
| `EVALUATION_ID` | int  | ✓        | The ID of the evaluation. |

**Options**

| Option            | Type      | Required | Description                                                                                                                     |
| ----------------- | --------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `--project`       | str       |          | Project ID. Env: `OUMI_PROJECT_ID`.                                                                                             |
| `--include-items` | flag      |          | Include per-item failure-mode detail in the response.                                                                           |
| `--limit`         | int range |          | Return at most N failure modes (default 100). The CLI fetches and loops over pages internally; you never pass a page token.     |
| `--all`           | flag      |          | Return every failure mode, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`. |
| `--max-items`     | int range |          | Hard safety cap for `--all` (default 10000).                                                                                    |

***

## GENERATE-TEMPLATE

Print a ready-to-edit evaluation recipe request body.

```bash theme={null}
oumi-cli evaluations generate-template [OPTIONS]
```

***

## GET

Show one evaluation by ID, with scores and per-evaluator detail.

```bash theme={null}
oumi-cli evaluations get [OPTIONS] {evaluation_id}
```

**Arguments**

| Argument        | Type | Required | Description                                             |
| --------------- | ---- | -------- | ------------------------------------------------------- |
| `EVALUATION_ID` | int  | ✓        | ID of the evaluation (see `oumi-cli evaluations list`). |

**Options**

| Option      | Type | Required | Description                                                                              |
| ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- |
| `--project` | str  |          | Project that owns the evaluation; falls back to OUMI\_PROJECT\_ID or the active profile. |

***

## LIST

List evaluations in a project, with optional filter, search, and sort.

```bash theme={null}
oumi-cli evaluations list [OPTIONS]
```

**Options**

| Option             | Type      | Required | Description                                                                                                                   |
| ------------------ | --------- | -------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `--project`        | str       |          | Project that owns the evaluation; falls back to OUMI\_PROJECT\_ID or the active profile.                                      |
| `--limit`          | int range |          | Return at most N evaluations (default 100). The CLI fetches and loops over pages internally; you never pass a page token.     |
| `--all`            | flag      |          | Return every evaluation, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`. |
| `--max-items`      | int range |          | Absolute cap on rows returned, in every mode (default 10000).                                                                 |
| `--search`         | str       |          | Keep only evaluations whose display name contains this substring.                                                             |
| `--sort-by`        | choice    |          | Field to sort by. One of: `created_at`, `updated_at`.                                                                         |
| `--sort-direction` | choice    |          | Sort order (asc is oldest-first).                                                                                             |

***

## RESULTS

Show per-row evaluation results, one verdict column per evaluator.

```bash theme={null}
oumi-cli evaluations results [OPTIONS] {evaluation_id}
```

**Arguments**

| Argument        | Type | Required | Description               |
| --------------- | ---- | -------- | ------------------------- |
| `EVALUATION_ID` | int  | ✓        | The ID of the evaluation. |

**Options**

| Option        | Type      | Required | Description                                                                                                                  |
| ------------- | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `--project`   | str       |          | Project ID. Env: `OUMI_PROJECT_ID`.                                                                                          |
| `--evaluator` | int       |          | Filter rows to a single evaluator's results (by evaluator ID); applied client-side after fetching.                           |
| `--id`        | str       |          | Filter to a single data item by its ID; applied client-side after fetching.                                                  |
| `--limit`     | int range |          | Return at most N data items (default 100). The CLI fetches and loops over pages internally; you never pass a page token.     |
| `--all`       | flag      |          | Return every data item, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`. |
| `--max-items` | int range |          | Hard safety cap for `--all` (default 10000).                                                                                 |

***

## RUN

Submit an evaluation from a recipe and wait for it to finish.

```bash theme={null}
oumi-cli evaluations run [OPTIONS]
```

**Options**

| Option                 | Type | Required | Description                                                                                              |
| ---------------------- | ---- | -------- | -------------------------------------------------------------------------------------------------------- |
| `--name`               | str  | ✓        | Human-readable display name for this evaluation run.                                                     |
| `--recipe`, `-r`       | str  | ✓        | Recipe ID (integer) or path to a JSON recipe file (recipeConfig.type must be 'evaluate').                |
| `--project`            | str  |          | Project to scope the operation to; falls back to OUMI\_PROJECT\_ID or the active profile.                |
| `--wait` / `--no-wait` | flag |          | Wait for the run to finish before returning (default); `--no-wait` returns the operation ID immediately. |

***