CALLING YOUR ENDPOINT

Your deployment’s endpoint is wire-compatible with the OpenAI and Anthropic APIs, so you don’t need an Oumi-specific client. Point your existing SDK at a different base URL, pass your deployment as the model, and the rest of your code stays the same.

BEFORE YOU START

You will need:

A deployment in the Active state. A deployment that is still Deploying or has Failed will not serve requests.
A personal Oumi API key. Use an existing key from User Settings > API Keys, or create one from the code-snippet panel on your deployment page.

Open your deployment and use the built-in code snippets rather than transcribing values by hand. They come pre-filled with your endpoint and model path, have a toggle for streaming, and can generate an API key inline.

ENDPOINT

	Value
OpenAI-compatible endpoint	`https://api.oumi.ai/inference/v1/chat/completions` — custom Oumi models, and external models served by OpenAI
Anthropic-compatible endpoint	`https://api.oumi.ai/inference/v1/messages` — external models served by Anthropic
Auth header	`X-API-Key: $OUMI_API_KEY`
`model` field	`projects/{project_id}/deployments/{deployment_id}`

Set your API key on the SDK client and it authenticates for you. You only need to set the header yourself for raw HTTP calls such as curl. The model field on the request body is a deployment resource path rather than a model name, for example projects/XFZ7K2/deployments/83. Copy the exact value from your deployment’s page.

OPENAI-COMPATIBLE REQUESTS

Use these for custom Oumi models and for external models served by OpenAI. Set base_url to https://api.oumi.ai/inference/v1 and pass your deployment as the model.

curl -X POST https://api.oumi.ai/inference/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $OUMI_API_KEY" \
  -d '{
    "model": "projects/{project_id}/deployments/{deployment_id}",
    "messages": [{"role": "user", "content": "your prompt"}]
  }'

from openai import OpenAI

client = OpenAI(
    api_key="<your-api-key>",
    base_url="https://api.oumi.ai/inference/v1",
)

response = client.chat.completions.create(
    model="projects/{project_id}/deployments/{deployment_id}",
    messages=[{"role": "user", "content": "your prompt"}],
)
print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.OUMI_API_KEY,
  baseURL: 'https://api.oumi.ai/inference/v1',
  defaultHeaders: { 'X-API-Key': process.env.OUMI_API_KEY ?? '' },
});

const response = await client.chat.completions.create({
  model: 'projects/{project_id}/deployments/{deployment_id}',
  messages: [{ role: 'user', content: 'your prompt' }],
});

ANTHROPIC-COMPATIBLE REQUESTS

External models served by Anthropic use this endpoint instead of /chat/completions, which rejects them. Note that max_tokens is required.

The Anthropic SDKs append /v1 to the base URL themselves, so set their base URL to https://api.oumi.ai/inference.

curl -X POST https://api.oumi.ai/inference/v1/messages \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $OUMI_API_KEY" \
  -d '{
    "model": "projects/{project_id}/deployments/{deployment_id}",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "your prompt"}]
  }'

import anthropic

client = anthropic.Anthropic(
    api_key="<your-api-key>",
    base_url="https://api.oumi.ai/inference",
)

client.messages.create(
    model="projects/{project_id}/deployments/{deployment_id}",
    max_tokens=1024,
    messages=[{"role": "user", "content": "your prompt"}],
)

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  apiKey: process.env.OUMI_API_KEY,
  baseURL: 'https://api.oumi.ai/inference',
});

await client.messages.create({
  model: 'projects/{project_id}/deployments/{deployment_id}',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'your prompt' }],
});

COLD STARTS

A custom Oumi model running with scale-to-zero has no warm replica while idle, so its first request after a quiet period spins up capacity before it responds. Expect that call to take noticeably longer than the ones that follow, and set generous client timeouts for it. If you need consistently low latency on every request, set min replicas to 1 so a replica is always warm. That deployment is then billed hourly whether or not it serves traffic. External models have no cold start, since the provider keeps its own capacity warm.

TRYING A DEPLOYMENT WITHOUT CODE

Open your deployment and use Try model to chat with it directly in the browser. It holds a multi-turn conversation, streams responses as they generate, and lets you set system instructions to test different prompts. This is the quickest way to confirm a single deployment behaves as expected before you wire it into an application. To compare models rather than exercise one, use the Model Playground instead. It puts your deployments and Oumi base models side by side against the same prompt, so you can weigh responses and latency across them at once. You can also ask the Oumi Agent to run a prompt against a deployment from chat, which returns the response inline along with a snippet you can reuse.

WHAT’S NEXT

Inference logs

Review the requests your deployment has served, including latency and token counts.

Quality monitoring

Score sampled live traffic with LLM judges and watch quality trends.

Getting started

Oumi workflow

CALLING YOUR ENDPOINT

BEFORE YOU START

ENDPOINT

OPENAI-COMPATIBLE REQUESTS

ANTHROPIC-COMPATIBLE REQUESTS

COLD STARTS

TRYING A DEPLOYMENT WITHOUT CODE

WHAT’S NEXT

Inference logs

Quality monitoring

​BEFORE YOU START

​ENDPOINT

​OPENAI-COMPATIBLE REQUESTS

​ANTHROPIC-COMPATIBLE REQUESTS

​COLD STARTS

​TRYING A DEPLOYMENT WITHOUT CODE

​WHAT’S NEXT

Inference logs

Quality monitoring

BEFORE YOU START

ENDPOINT

OPENAI-COMPATIBLE REQUESTS

ANTHROPIC-COMPATIBLE REQUESTS

COLD STARTS

TRYING A DEPLOYMENT WITHOUT CODE

WHAT’S NEXT