WHEN TO SELF-HOST
An Oumi deployment is the faster path for most teams, and it is the only path with inference logs, serving metrics, and quality monitoring in Oumi. Self-host when a requirement rules it out:- Your data can’t leave your own infrastructure, or your network is air-gapped.
- The model must fit into a serving stack you already run.
- You want to run the model on hardware you already own.
EXPORT A MODEL
- On the Models page, open the model’s actions menu and select
Export. - For a model trained with LoRA, choose the Format:
Adapter onlyexports the small LoRA adapter. Serve it together with the base model. Choose it when your serving stack loads adapters, or to serve several adapters on one base model.Merged full weightsmerges the adapter into the base model and exports a standalone model. Choose it when your serving stack needs a single model. Merging can take a few hours, and you must keep the browser tab open until it finishes.
- Click
Continue Export, and download the archive.
SERVE THE MODEL WITH VLLM
vLLM runs an OpenAI-compatible server from the exported files. Install it, and unpack the archive:OTHER PLATFORMS
An exported model also works with other serving options:- Hugging Face
transformers: Load the model in Python, or serve it with thetransformersCLI. See its serving documentation. - Amazon Bedrock: Import the model as a Bedrock custom model. See the AWS walkthrough.