Skip to main content
A deployment is an inference endpoint that lives in your project. Your application calls it with the OpenAI or Anthropic SDKs it already uses, and Oumi handles what sits behind it. There is no model packaging, no GPU provisioning, and no serving stack to maintain. The Oumi Agent can set this up from chat. Tell it which of your models to deploy and it asks you to approve the cost before provisioning anything, then hands back a runnable snippet.

​
DEPLOYMENT TYPES

A deployment serves either a model you built or one from a third-party provider. Both are created from the same dialog, called the same way, and logged the same way. They differ only in what runs behind the endpoint. Learn how to deploy a model →

​
THE DEPLOYMENT LOOP

Once an endpoint is live, the same cycle applies whichever kind you chose:
  1. Deploy the model. Start from the Models or Deployments page. You can also deploy your own models automatically when training finishes.
  2. Call the endpoint. Point the OpenAI or Anthropic SDK at your deployment.
  3. Review inference logs. See what users are asking the model in production.
  4. Monitor health and quality. Health metrics track traffic, errors, and latency, while quality monitoring scores a sample of production traffic with LLM judges.
  5. Improve the model. Use compounding intelligence to turn monitoring insights into targeted training data, then retrain and redeploy.

​
MONITORING

Once traffic flows, the deployment reports on itself from two angles. The Health tab tracks the operational side: request volume, error rate, latency percentiles, and token throughput. The Insights tab tracks quality: attach judge evaluators to score a sample of live traffic, and their score trends show whether responses are holding up in production. When a trend dips, drill into the scored requests behind it or run a failure modes analysis directly on that slice of traffic. Learn about quality monitoring →

​
COMPARE MODELS SIDE BY SIDE

Before you point an application at one endpoint, put the candidates next to each other. The Model Playground chats with up to four deployments or Oumi base models at once, sending the same prompt to all of them, so you can read the answers together and watch how latency and token usage differ. Learn about the Model Playground →

​
SELF-HOSTING

If a hosted endpoint cannot meet your requirements, export your model and serve it on your own hardware or a cloud GPU instance instead. Inference logging and monitoring are features of the platform endpoint, so a self-hosted model does not report back into Oumi. Learn about self-hosting →

​
WHAT’S NEXT

Deploy a model

Stand up a managed endpoint, for your own model or an external one.

Calling your endpoint

Call your deployment from Python, TypeScript, or curl.

Quality monitoring

Score sampled production traffic with LLM judges and watch quality trends.

Self-hosting

Export your model and serve it on your own infrastructure.