| Command | Description |
|---|---|
attach | Attach a deployment to a monitor. |
create | Create a monitor that scores sampled deployment traffic. |
delete | Delete Monitor. |
detach | Detach a deployment from a monitor. |
failure-modes generate | Run an LLM failure-mode analysis over a deployment target’s scored logs. |
failure-modes runs list | List past failure-mode analyses of a monitor target’s scored logs. |
get | Get Monitor. |
insights | Get per-judge score insights for a deployment target. |
list | List Monitors. |
scores | Fetch score detail for the logs a monitor scored. |
update | Update a monitor’s judges, sample rate, or name (at least one is required). |
ATTACH
Attach a deployment to a monitor.oumi-cli monitors attach [OPTIONS] {monitor_id} {deployment_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The ID of the deployment. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. |
CREATE
Create a monitor that scores sampled deployment traffic.oumi-cli monitors create [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--judge | int | ✓ | Evaluator to judge sampled traffic with, at its latest version; repeat --judge to add several. |
--sample-rate | float | ✓ | Fraction of traffic to score, between 0 and 1. |
--display-name | str | Human-readable name for the monitor. |
DELETE
Delete Monitor.oumi-cli monitors delete [OPTIONS] {monitor_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--force | flag | Skip confirmation prompt. | |
--yes | flag | Skip confirmation prompt (alias for --force). |
DETACH
Detach a deployment from a monitor.oumi-cli monitors detach [OPTIONS] {monitor_id} {deployment_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The ID of the deployment. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--force | flag | Skip confirmation prompt. | |
--yes | flag | Skip confirmation prompt (alias for --force). |
FAILURE-MODES GENERATE
Run an LLM failure-mode analysis over a deployment target’s scored logs.oumi-cli monitors failure-modes generate [OPTIONS] {monitor_id} {deployment_id} {target_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The public ID of the deployment. |
TARGET_ID | int | ✓ | The public ID of the deployment target. |
| Option | Type | Required | Description |
|---|---|---|---|
--time-window | parse_time_window | Analyze the logs scored in the last DURATION instead of an explicit --start/--end, e.g. ‘1h’, ‘30m’, ‘7d’. Mutually exclusive with --start/--end. Defaults to 1 hour when none of --time-window/--start/--end is given. | |
--end | parse_iso_timestamp | End of the scored-logs window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to now. Mutually exclusive with --time-window. | |
--start | parse_iso_timestamp | Start of the scored-logs window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to 1 hour before --end. Mutually exclusive with --time-window. | |
--display-name | str | Display name for the analysis run this creates. | |
--skip-failure-modes | flag | Materialize the evaluation of the window’s logs but stop before deriving failure modes from it. Leaves a run oumi-cli evaluations failure-modes generate can analyze later. | |
--project | str | Project that owns the monitor; falls back to OUMI_PROJECT_ID or the active profile. | |
--wait / --no-wait | flag | Wait for the analysis to finish before returning (default); --no-wait returns the operation ID immediately. |
FAILURE-MODES RUNS LIST
List past failure-mode analyses of a monitor target’s scored logs.oumi-cli monitors failure-modes runs list [OPTIONS] {monitor_id} {deployment_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The public ID of the deployment. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project that owns the monitor; falls back to OUMI_PROJECT_ID or the active profile. | |
--limit | int range | Return at most N items (default 100). The CLI loops over pages internally; you never pass a page token. | |
--all | flag | Return everything, looping until the cursor is exhausted (bounded by --max-items). Mutually exclusive with --limit. | |
--max-items | int range | Hard safety cap for --all (default 10000). |
GET
Get Monitor.oumi-cli monitors get [OPTIONS] {monitor_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. |
INSIGHTS
Get per-judge score insights for a deployment target.oumi-cli monitors insights [OPTIONS] {monitor_id} {deployment_id} {target_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The public ID of the deployment. |
TARGET_ID | int | ✓ | The public ID of the deployment target. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--time-window | parse_time_window | Fetch the last DURATION instead of an explicit --start/--end, e.g. ‘1h’, ‘30m’, ‘7d’. Mutually exclusive with --start/--end. Defaults to 1 hour when none of --time-window/--start/--end is given. | |
--end | parse_iso_timestamp | End of the window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to now. Mutually exclusive with --time-window. | |
--start | parse_iso_timestamp | Start of the window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to 1 hour before --end. Mutually exclusive with --time-window. |
LIST
List Monitors.oumi-cli monitors list [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--limit | int range | Return at most N items (default 100). The CLI loops over pages internally; you never pass a page token. | |
--all | flag | Return everything, looping until the cursor is exhausted (bounded by --max-items). Mutually exclusive with --limit. | |
--max-items | int range | Hard safety cap for --all (default 10000). |
SCORES
Fetch score detail for the logs a monitor scored.oumi-cli monitors scores [OPTIONS] {monitor_id} {deployment_id} {target_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
DEPLOYMENT_ID | int | ✓ | The public ID of the deployment. |
TARGET_ID | int | ✓ | The public ID of the deployment target. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--time-window | parse_time_window | Fetch score detail for every log scored in the last DURATION, e.g. ‘1h’, ‘30m’, ‘7d’. Mutually exclusive with --start/--end and --score-log-id. Defaults to 5 minutes when no id and no window is given. | |
--end | parse_iso_timestamp | End of the window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to now. Mutually exclusive with --time-window and --score-log-id. | |
--start | parse_iso_timestamp | Start of the window (inclusive), e.g. 2024-01-01T00:00:00. Defaults to 5 minutes before --end. Mutually exclusive with --time-window and --score-log-id. | |
--score-log-id | str | Score log id to fetch detail for; repeat to fetch several — ids come from the ‘Score Log’ column of monitors insights. Fetched regardless of age; mutually exclusive with --time-window/--start/--end. Default: []. |
UPDATE
Update a monitor’s judges, sample rate, or name (at least one is required).oumi-cli monitors update [OPTIONS] {monitor_id}
| Argument | Type | Required | Description |
|---|---|---|---|
MONITOR_ID | int | ✓ | The ID of the monitor. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | str | Project ID. Env: OUMI_PROJECT_ID. | |
--judge | int | Replace the monitor’s judges with these evaluators, at their latest versions; repeat --judge to give several. | |
--sample-rate | float | Fraction of traffic to score, between 0 and 1. | |
--display-name | str | Human-readable name for the monitor. |