| Command | Description |
|---|---|
analysis download | Download a completed analysis’s artifacts to local disk. |
analysis get | Show a completed analysis’s results in place. |
analysis list | List completed quality analyses for a dataset. |
delete | Permanently delete a dataset from the project. |
download | Download a dataset version to local disk. |
from-failure-modes | Generate an improved dataset from an evaluation’s failure modes, waiting for it to finish. |
generate-responses | Generate model responses for every conversation in a dataset, with optional tool calling. |
get | Show metadata for a single dataset. |
import-logs | Import a LangFuse or LangSmith trace export as a new dataset. |
items bulk-delete | Permanently delete the rows a prior analysis’s failing tests flagged. |
items get | Show a single row (data item) from a dataset. |
items list | List the rows (data items) in a dataset, optionally filtered. |
list | List datasets in a project, optionally filtered and sorted. |
sample | Randomly sample rows from a dataset into a new dataset; the source is unchanged. |
supported-models | List models suggested for dataset synthesis, before authoring a synthesize recipe. |
synthesis preview | Synthesize a few samples from a recipe without creating a dataset. |
synthesize | Synthesize a new dataset from a recipe, waiting for it to finish. |
update | Rename a dataset. |
upload | Upload a local file as a new dataset or a new version of an existing one. |
versions delete | Permanently delete one version of a dataset. |
versions get | Show metadata for a single dataset version. |
versions list | List versions of a dataset, newest first by default. |
versions restore | Restore a prior version by creating a new version with that version’s content. |
ANALYSIS DOWNLOAD
Download a completed analysis’s artifacts to local disk.oumi-cli datasets analysis download [OPTIONS] DATASET_ID ANALYSIS_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset that owns the analysis (see oumi-cli datasets list). |
ANALYSIS_ID | integer | ✓ | ID of the analysis (see oumi-cli datasets analysis list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--format | choice | Artifact format to download. Default: jsonl. | |
--out | path | Destination file or directory. Defaults to ./analysis-<dataset_id>-<analysis_id>.<ext> in the current directory. | |
--expiration | integer range | Lifetime of the download’s presigned URL, in seconds (max 604800, 7 days). Default: 3600. |
ANALYSIS GET
Show a completed analysis’s results in place.oumi-cli datasets analysis get [OPTIONS] DATASET_ID ANALYSIS_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset that owns the analysis (see oumi-cli datasets list). |
ANALYSIS_ID | integer | ✓ | ID of the analysis (see oumi-cli datasets analysis list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. |
ANALYSIS LIST
List completed quality analyses for a dataset.oumi-cli datasets analysis list [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset whose analyses to list (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-version | integer | Filter to analyses of this dataset version. Omit to see analyses across every version. | |
--limit | integer range | Maximum number of analyses to return (default 100). | |
--all | flag | Return all analyses, following pages until none are left (capped by --max-items). Cannot be combined with --limit. | |
--max-items | integer range | Cap on how many analyses --all returns (default 10000). |
DELETE
Permanently delete a dataset from the project.oumi-cli datasets delete [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--force | flag | Skip the confirmation prompt. Without a TTY or under --no-input, this (or --yes) is required; otherwise the command exits 2 without changing anything. | |
--yes | flag | Same as --force. |
DOWNLOAD
Download a dataset version to local disk.oumi-cli datasets download [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-version | integer | ID of the version to download. Defaults to the latest version. | |
--out | path | Destination file or directory. Defaults to ./dataset-<id>-<ver>.<ext> in the current directory. | |
--expiration | integer range | Lifetime of the download’s presigned URL, in seconds (max 604800, 7 days). Default: 3600. |
FROM-FAILURE-MODES
Generate an improved dataset from an evaluation’s failure modes, waiting for it to finish.oumi-cli datasets from-failure-modes [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--evaluation | integer | ✓ | ID of the evaluation whose failure modes to generate improved samples from (see oumi-cli evaluations list). |
--model | text | ✓ | Name of the model that generates the improved samples, exactly as oumi-cli datasets supported-models lists it (e.g. ‘claude-sonnet-5’, ‘Qwen/Qwen2.5-7B-Instruct’). |
--failure-mode | integer | Failure mode ID to draw items from (repeatable); omit to use every failure mode on the evaluation. See oumi-cli evaluations failure-modes list. | |
--item | text | Data item ID to generate a sample for (repeatable); omit to use every item in the selected failure modes. See ‘evaluations failure-modes get-items <fm-id> --output id’. | |
--samples-per-item | integer range | Number of improved samples to generate per data item. Default: 1. | |
--provider | text | Provider serving --model (‘openai’, ‘anthropic’, ‘oumi’, …). Only needed when one name is served by more than one provider. | |
--temperature | float range | Sampling temperature for the generated samples. | |
--max-tokens | integer range | Maximum number of new tokens to generate per sample. | |
--seed | integer | Seed for inference determinism. | |
--requests-per-minute | integer range | Cap on requests per minute sent to the provider. | |
--tool-implementation | integer | ID of a tool implementation the model may call while generating (repeatable); passing any enables multi-step tool calling. Run oumi-cli tools implementations list <TOOL_ID> for the IDs to put here. | |
--user-model | text | Name of the model that writes the simulated user’s replies, exactly as oumi-cli datasets supported-models lists it; passing one enables multi-turn conversations. Must be an API-served model, not a pretrained one. | |
--user-model-provider | text | Provider serving --user-model. Only needed when one name is served by more than one provider. | |
--max-user-turns | integer range | Maximum number of user messages in a generated conversation, including the first. Needs --user-model; the API defaults it to 20. | |
--dataset | integer | ID of an existing dataset to add the generated samples to; omit to create a new dataset (see oumi-cli datasets list). | |
--train | float range | Fraction of the new dataset assigned to the train split. Pass --train/--validation/--test together, summing to 1.0, or omit all three; not accepted with --dataset, which brings its own split. | |
--validation | float range | Fraction of the new dataset assigned to the validation split. | |
--test | float range | Fraction of the new dataset assigned to the test split. | |
--display-name | text | Display name for the output dataset. Auto-generated if omitted. | |
--project | text | Project to generate the dataset in; falls back to OUMI_PROJECT_ID or the active profile. | |
--wait / --no-wait | flag | Wait for generation to finish (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. |
GENERATE-RESPONSES
Generate model responses for every conversation in a dataset, with optional tool calling.oumi-cli datasets generate-responses [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--dataset | integer | ID of the dataset to generate responses for (see oumi-cli datasets list). Required unless the body is passed with --input-json or --input-json-file. | |
--model | text | Name of the model that writes the responses, exactly as oumi-cli datasets supported-models --include-gpu-models lists it (e.g. ‘claude-sonnet-5’, ‘Qwen/Qwen2.5-7B-Instruct’). Required unless the body is passed with --input-json or --input-json-file. | |
--provider | text | Provider serving --model (‘openai’, ‘anthropic’, ‘oumi’, …). Only needed when one name is served by more than one provider. | |
--display-name | text | Display name for the output dataset. Auto-generated if omitted. | |
--temperature | float range | Sampling temperature for the responses. | |
--requests-per-minute | integer range | Cap on requests per minute sent to the provider. | |
--system-instruction | text | System prompt prepended to every request. | |
--tool-implementation | integer | ID of a tool implementation the model may call during the run (see oumi-cli tools implementations list <TOOL_ID>); repeat for several. Passing any turns one response per conversation into a multi-step tool-calling rollout. | |
--user-model | text | Name of the model that writes the next user turn, from the same oumi-cli datasets supported-models catalog as --model. Passing it makes the run multi-turn. Must be an API-served model, not a GPU one. | |
--user-model-provider | text | Provider serving --user-model. Only needed when one name is served by more than one provider. | |
--max-user-turns | integer range | Cap on the user messages in one conversation, counting the first (default 20). Requires --user-model; the run can still end a conversation earlier. | |
--input-json | text | Whole request body as inline JSON, in place of the flags above. Pass at most one of --input-json or --input-json-file. | |
--input-json-file | path | Path to a file holding the whole request body as JSON, in place of the flags above. | |
--project | text | Project to generate responses in; falls back to OUMI_PROJECT_ID or the active profile. | |
--wait / --no-wait | flag | Wait for generation to finish (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. |
GET
Show metadata for a single dataset.oumi-cli datasets get [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. |
IMPORT-LOGS
Import a LangFuse or LangSmith trace export as a new dataset.oumi-cli datasets import-logs [OPTIONS] FILE
| Argument | Type | Required | Description |
|---|---|---|---|
FILE | path | ✓ | Path to the local .jsonl trace export (a LangFuse Tracing-tab export or a LangSmith run export). |
| Option | Type | Required | Description |
|---|---|---|---|
--platform | choice | ✓ | Observability platform the export came from. One of: langfuse, langsmith. |
--display-name | text | ✓ | Display name for the new dataset. |
--project | text | Project to import the dataset into; falls back to OUMI_PROJECT_ID or the active profile. | |
--dedupe / --no-dedupe | flag | Drop prefix-overlap rows where one trace’s full conversation is the prior turns of a longer trace in the same export. | |
--wait / --no-wait | flag | Wait for the dataset ingest to finish (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. | |
--dry-run | flag | Validate the file as JSONL and preview its row count locally; makes no API calls and imports nothing. |
ITEMS BULK-DELETE
Permanently delete the rows a prior analysis’s failing tests flagged.oumi-cli datasets items bulk-delete [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--analysis-test | integer | ✓ | Delete rows flagged by this analysis’s failing tests (its test_affected_rows artifact). |
--test-id | text | Test ID whose flagged rows should be deleted (repeatable); at least one is required. See oumi-cli datasets analysis get. | |
--create-new-dataset | flag | Write the surviving rows to a new dataset instead of deleting in place; the source dataset is left unchanged. | |
--new-dataset-name | text | Display name for the new dataset (requires --create-new-dataset); defaults to ‘Copy of {source name}’. | |
--wait / --no-wait | flag | Wait for the new dataset to finish being written (default; only meaningful with --create-new-dataset); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. | |
--force | flag | Skip the confirmation prompt. Without a TTY or under --no-input, this (or --yes) is required; otherwise the command exits 2 without changing anything. | |
--yes | flag | Same as --force. |
ITEMS GET
Show a single row (data item) from a dataset.oumi-cli datasets items get [OPTIONS] DATASET_ID DATAITEM_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
DATAITEM_ID | text | ✓ | ID of the row (see oumi-cli datasets items list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-version | integer | ID of the dataset version to read the row from. Defaults to the latest version. |
ITEMS LIST
List the rows (data items) in a dataset, optionally filtered.oumi-cli datasets items list [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-version | integer | ID of the dataset version to list rows from. Defaults to the latest version. | |
--search | text | Case-insensitive substring filter applied to each row’s content. | |
--random-offset | flag | Start from a random offset instead of the beginning, useful for sampling a few rows without --all. Ignored when --search is set. | |
--limit | integer range | Maximum number of rows to return (default 100). | |
--all | flag | Return all rows, following pages until none are left (capped by --max-items). Cannot be combined with --limit. | |
--max-items | integer range | Cap on how many rows --all returns (default 10000). |
LIST
List datasets in a project, optionally filtered and sorted.oumi-cli datasets list [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project to list datasets from; falls back to OUMI_PROJECT_ID or the active profile. | |
--limit | integer range | Maximum number of datasets to return (default 100). | |
--all | flag | Return all datasets, following pages until none are left (capped by --max-items). Cannot be combined with --limit. | |
--max-items | integer range | Cap on how many datasets --all returns (default 10000). | |
--search | text | Only show datasets whose name matches this search text. | |
--sort-by | choice | Field to sort by. One of: created_at, updated_at. | |
--sort-direction | choice | Sort order: asc is oldest-first (default desc). |
SAMPLE
Randomly sample rows from a dataset into a new dataset; the source is unchanged.oumi-cli datasets sample [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset to sample (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--size | integer range | ✓ | Number of rows to sample; must not exceed the version’s row count. |
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-version | integer range | ID of the dataset version to sample. Defaults to the latest version. | |
--seed | integer range | Random seed; the same seed on the same version picks the same rows. Random if omitted. | |
--display-name | text | Display name for the new dataset. | |
--description | text | Description for the new dataset’s first version. | |
--wait / --no-wait | flag | Wait for the new dataset to finish being written (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. |
SUPPORTED-MODELS
List models suggested for dataset synthesis, before authoring asynthesize recipe.
oumi-cli datasets supported-models [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project to scope the lookup to; falls back to OUMI_PROJECT_ID or the active profile. | |
--include-gpu-models | flag | Also list GPU-served models (GPU=true), which generate-responses accepts and synthesize does not. |
SYNTHESIS PREVIEW
Synthesize a few samples from a recipe without creating a dataset.oumi-cli datasets synthesis preview [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--recipe, -r | text | ✓ | Recipe ID (see oumi-cli recipes list), or path to a JSON synthesis recipe file (see oumi-cli recipes generate-template --type synthesis). |
--sample-size | integer range | How many samples to synthesize (1-20). Default: 5. | |
--project | text | Project to preview in; falls back to OUMI_PROJECT_ID or the active profile. |
SYNTHESIZE
Synthesize a new dataset from a recipe, waiting for it to finish.oumi-cli datasets synthesize [OPTIONS]
| Option | Type | Required | Description |
|---|---|---|---|
--recipe, -r | text | ✓ | Recipe ID (see oumi-cli recipes list), or path to a JSON synthesis recipe file (see oumi-cli recipes generate-template --type synthesis). |
--display-name | text | Display name for the synthesized dataset. Auto-generated if omitted. | |
--project | text | Project to synthesize in; falls back to OUMI_PROJECT_ID or the active profile. | |
--wait / --no-wait | flag | Wait for synthesis to finish (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. |
UPDATE
Rename a dataset.oumi-cli datasets update [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--display-name | text | New display name for the dataset. |
UPLOAD
Upload a local file as a new dataset or a new version of an existing one.oumi-cli datasets upload [OPTIONS] FILE
| Argument | Type | Required | Description |
|---|---|---|---|
FILE | path | ✓ | Path to the local JSONL dataset file (one JSON object per line, max 5 GB); see oumi-cli datasets --help for the row format, including tool calls. |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project to upload the dataset into; falls back to OUMI_PROJECT_ID or the active profile. | |
--dataset-id | integer | Upload as a new version of this existing dataset. Omit to create a brand-new dataset. | |
--display-name | text | Display name for the dataset. Defaults to the local filename. | |
--expiration | integer | Lifetime of the upload’s presigned URL, in seconds. Default: 3600. | |
--wait / --no-wait | flag | Wait for the dataset ingest to finish (default); --no-wait returns the operation ID immediately; follow with oumi-cli operations wait <id>. | |
--dry-run | flag | Validate the file as JSONL and preview its row count and detected schema type; makes no API calls and uploads nothing. |
VERSIONS DELETE
Permanently delete one version of a dataset.oumi-cli datasets versions delete [OPTIONS] DATASET_ID VERSION_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
VERSION_ID | integer | ✓ | ID of the version (see oumi-cli datasets versions list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--force | flag | Skip the confirmation prompt. Without a TTY or under --no-input, this (or --yes) is required; otherwise the command exits 2 without changing anything. | |
--yes | flag | Same as --force. |
VERSIONS GET
Show metadata for a single dataset version.oumi-cli datasets versions get [OPTIONS] DATASET_ID VERSION_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
VERSION_ID | integer | ✓ | ID of the version (see oumi-cli datasets versions list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. |
VERSIONS LIST
List versions of a dataset, newest first by default.oumi-cli datasets versions list [OPTIONS] DATASET_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset whose versions to list (see oumi-cli datasets list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--limit | integer range | Maximum number of versions to return (default 100). | |
--all | flag | Return all versions, following pages until none are left (capped by --max-items). Cannot be combined with --limit. | |
--max-items | integer range | Cap on how many versions --all returns (default 10000). | |
--sort-by | choice | Field to sort by. One of: created_at, updated_at, version_id. | |
--sort-direction | choice | Sort order: asc is oldest-first (default desc). |
VERSIONS RESTORE
Restore a prior version by creating a new version with that version’s content.oumi-cli datasets versions restore [OPTIONS] DATASET_ID VERSION_ID
| Argument | Type | Required | Description |
|---|---|---|---|
DATASET_ID | integer | ✓ | ID of the dataset (see oumi-cli datasets list). |
VERSION_ID | integer | ✓ | ID of the version (see oumi-cli datasets versions list). |
| Option | Type | Required | Description |
|---|---|---|---|
--project | text | Project that owns the dataset; falls back to OUMI_PROJECT_ID or the active profile. | |
--display-name | text | Display name for the new version. Defaults to ‘Restore to v<version_id>’. | |
--description | text | Description for the new version. |