> ## Documentation Index > Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt > Use this file to discover all available pages before exploring further. # DEPLOYMENTS > Serve models behind an Oumi inference endpoint | Command | Description | | ----------------------------------------- | --------------------------------------------------------------------------------- | | [`create`](#create) | Create a new model deployment from a JSON request body. | | [`delete`](#delete) | Delete a deployment by ID. | | [`endpoint`](#endpoint) | Print the inference endpoint, model path, and a runnable curl snippet. | | [`generate-template`](#generate-template) | Print a ready-to-edit CreateDeploymentRequest body for the given deployment type. | | [`get`](#get) | Get a deployment by ID. | | [`invoke`](#invoke) | Run one inference call against a deployment and print the reply. | | [`is-ready`](#is-ready) | Check whether a deployment has an active replica available to serve inference. | | [`list`](#list) | List deployments in the project. | | [`logs delete`](#logs-delete) | Delete inference log entries for a deployment, optionally scoped to a time range. | | [`logs export`](#logs-export) | Export a time range of inference logs into a new or existing dataset. | | [`logs get`](#logs-get) | Get a single inference log entry by ID, including the full request/response. | | [`logs list`](#logs-list) | List inference log entries for a deployment. | | [`supported-models`](#supported-models) | List the models this project can deploy, before `create`/`generate-template`. | | [`update`](#update) | Update a deployment by ID. | | [`wake-up`](#wake-up) | Send a dummy request to scale a provider-idled deployment back up. | *** ## CREATE Create a new model deployment from a JSON request body. ```bash theme={null} oumi-cli deployments create [OPTIONS] ``` **Options** | Option | Type | Required | Description | | ---------------------- | ---- | -------- | ---------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project to create the deployment in; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--input-json` | str | | Request body as an inline JSON string. Pass exactly one of `--input-json` or `--input-json-file`. | | `--input-json-file` | path | | Path to a file holding the request body as JSON. Pass exactly one of `--input-json` or `--input-json-file`. | | `--wait` / `--no-wait` | flag | | Wait for the deployment to become active before returning (default); `--no-wait` returns the operation ID immediately. | *** ## DELETE Delete a deployment by ID. ```bash theme={null} oumi-cli deployments delete [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ------------------------ | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID to act on. | **Options** | Option | Type | Required | Description | | ---------------------- | ---- | -------- | --------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--force` | flag | | Skip confirmation prompt. | | `--yes` | flag | | Skip confirmation prompt (alias for `--force`). | | `--wait` / `--no-wait` | flag | | Wait for the deployment to be torn down before returning (default); `--no-wait` returns the operation ID immediately. | *** ## ENDPOINT Print the inference endpoint, model path, and a runnable curl snippet. ```bash theme={null} oumi-cli deployments endpoint [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | -------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID. | **Options** | Option | Type | Required | Description | | ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | *** ## GENERATE-TEMPLATE Print a ready-to-edit CreateDeploymentRequest body for the given deployment type. ```bash theme={null} oumi-cli deployments generate-template [OPTIONS] ``` **Options** | Option | Type | Required | Description | | -------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------ | | `--type` | choice | ✓ | Deployment kind to generate a template for; sets modelSpec.type in the output. One of: `custom`, `external`. | *** ## GET Get a deployment by ID. ```bash theme={null} oumi-cli deployments get [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ------------------------ | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID to act on. | **Options** | Option | Type | Required | Description | | ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | *** ## INVOKE Run one inference call against a deployment and print the reply. ```bash theme={null} oumi-cli deployments invoke [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | -------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID. | **Options** | Option | Type | Required | Description | | -------------------------- | ----------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--prompt`, `-p` | str | ✓ | Prompt text ('-' reads from stdin). | | `--system` | str | | System instructions. | | `--temperature` | float range | | Sampling temperature. | | `--max-tokens` | int range | | Max output tokens. | | `--top-p` | float range | | Nucleus sampling top-p. | | `--stream` / `--no-stream` | flag | | Stream tokens (default: on a TTY). | | `--wait` / `--no-wait` | flag | | Wake a scaled-to-zero deployment and wait for it before sending the prompt (default); `--no-wait` fails with exit 9 instead of waiting. | | `--timeout` | int range | | Seconds the CLI will wait: the wake-up wait and the wait for each response chunk share this one budget, rather than each getting it in full. Bounds waiting only, not connect/write. 0 waits indefinitely (bounded only by Ctrl-C). Default: `600`. | *** ## IS-READY Check whether a deployment has an active replica available to serve inference. ```bash theme={null} oumi-cli deployments is-ready [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ----------------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID to check. | **Options** | Option | Type | Required | Description | | ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | *** ## LIST List deployments in the project. ```bash theme={null} oumi-cli deployments list [OPTIONS] ``` **Options** | Option | Type | Required | Description | | ------------- | --------- | -------- | ----------------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project to list deployments in; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--limit` | int range | | Return at most N deployments (default 100). The CLI fetches and loops over pages internally; you never pass a page token. | | `--all` | flag | | Return every deployment, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`. | | `--max-items` | int range | | Hard safety cap for `--all` (default 10000). | *** ## LOGS DELETE Delete inference log entries for a deployment, optionally scoped to a time range. ```bash theme={null} oumi-cli deployments logs delete [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ----------------------------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID whose logs to delete. | **Options** | Option | Type | Required | Description | | -------------- | -------- | -------- | ---------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--start-time` | datetime | | Delete logs at or after this time (e.g. 2024-01-01T00:00:00). | | `--end-time` | datetime | | Delete logs at or before this time. | | `--force` | flag | | Skip confirmation prompt. | | `--yes` | flag | | Skip confirmation prompt (alias for `--force`). | *** ## LOGS EXPORT Export a time range of inference logs into a new or existing dataset. ```bash theme={null} oumi-cli deployments logs export [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ----------------------------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID whose logs to export. | **Options** | Option | Type | Required | Description | | ---------------------- | -------- | -------- | ----------------------------------------------------------------------------------------------------------- | | `--start-time` | datetime | ✓ | Start of the time range to export, inclusive (e.g. 2024-01-01T00:00:00). | | `--end-time` | datetime | ✓ | End of the time range to export, inclusive. | | `--dataset-id` | int | | Append the exported logs to this existing dataset. Mutually exclusive with `--dataset-name`. | | `--dataset-name` | str | | Create a new dataset with this name from the exported logs. Mutually exclusive with `--dataset-id`. | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--wait` / `--no-wait` | flag | | Wait for the export to finish before returning (default); `--no-wait` returns the operation ID immediately. | *** ## LOGS GET Get a single inference log entry by ID, including the full request/response. ```bash theme={null} oumi-cli deployments logs get [OPTIONS] {deployment_id} {log_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | -------------------------------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID that owns the log entry. | | `LOG_ID` | str | ✓ | ULID of the log entry to fetch. | **Options** | Option | Type | Required | Description | | ----------- | ---- | -------- | ---------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | *** ## LOGS LIST List inference log entries for a deployment. ```bash theme={null} oumi-cli deployments logs list [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | --------------------------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID whose logs to list. | **Options** | Option | Type | Required | Description | | -------------- | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--start-time` | datetime | | Only include logs at or after this time (e.g. 2024-01-01T00:00:00). | | `--end-time` | datetime | | Only include logs at or before this time. | | `--limit` | int range | | Return at most N log entries (default 100). The CLI fetches and loops over pages internally; you never pass a page token. | | `--all` | flag | | Return every log entry, looping until the cursor is exhausted (bounded by `--max-items`). Mutually exclusive with `--limit`. | | `--max-items` | int range | | Hard safety cap for `--all` (default 10000). | *** ## SUPPORTED-MODELS List the models this project can deploy, before `create`/`generate-template`. ```bash theme={null} oumi-cli deployments supported-models [OPTIONS] ``` **Options** | Option | Type | Required | Description | | ----------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--source` | choice | | Limit to one source: 'custom' (this project's tuned model versions) or 'external' (curated provider catalog, e.g. OpenAI/Anthropic). Omit to list both. | | `--project` | str | | Project to scope the lookup to; falls back to OUMI\_PROJECT\_ID or the active profile. | *** ## UPDATE Update a deployment by ID. ```bash theme={null} oumi-cli deployments update [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | ------------------------ | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID to act on. | **Options** | Option | Type | Required | Description | | ---------------------------------------------------------------- | --------- | -------- | --------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--display-name` | str | | New display name for the deployment. | | `--inference-logging-enabled` / `--no-inference-logging-enabled` | flag | | Enable or disable inference request logging for the deployment. | | `--min-replicas` | int range | | Minimum replica count for autoscaling. Must be passed together with `--max-replicas`. | | `--max-replicas` | int range | | Maximum replica count for autoscaling. Must be passed together with `--min-replicas`. | | `--scale-down-idle-seconds` | int range | | Idle time before scaling down to `--min-replicas`. Requires `--min-replicas`/`--max-replicas` in the same invocation. | *** ## WAKE-UP Send a dummy request to scale a provider-idled deployment back up. ```bash theme={null} oumi-cli deployments wake-up [OPTIONS] {deployment_id} ``` **Arguments** | Argument | Type | Required | Description | | --------------- | ---- | -------- | -------------- | | `DEPLOYMENT_ID` | int | ✓ | Deployment ID. | **Options** | Option | Type | Required | Description | | ---------------------- | --------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `--project` | str | | Project that owns the deployment; falls back to OUMI\_PROJECT\_ID or the active profile. | | `--wait` / `--no-wait` | flag | | Wait for the deployment to be ready to serve inference before returning (default); `--no-wait` returns as soon as the wake-up request is delivered. | | `--timeout` | int range | | Seconds to wait for readiness under `--wait`; 0 waits indefinitely (bounded only by Ctrl-C). Default: `600`. | ***