> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# SERVING METRICS

> Read a deployment's traffic, errors, latency, and throughput, and act on what they show

Serving metrics show whether a deployment serves requests reliably and quickly. They come from the [inference logs](https://docs.oumi.ai/guides/deployment/inference-logs.md), so every deployment with logging on reports them, with no metrics stack to set up. Serving metrics tell you whether the deployment works. To learn whether its answers are good, use [quality monitoring](https://docs.oumi.ai/guides/deployment/monitoring.md).

You can also ask [FAiE](https://docs.oumi.ai/guides/intro/faie.md) (Frontier AI Engineer Agent) how a deployment is doing. It reads the same metrics, and with monitoring on, it can tell you whether a problem is operational or a drop in quality.

## WHAT EACH METRIC MEASURES

The deployment's **Health** tab shows these metrics for the **Time window** you choose.

![Health tab with the summary metrics and the Requests, Request Latency, Time to First Token, Token Throughput, and Error Rate charts](https://mintcdn.com/oumi/D0vKyXNqx2ohrLV9/images/deployment-health.jpg?fit=max\&auto=format\&n=D0vKyXNqx2ohrLV9\&q=85\&s=2052209abf488f7aaf5f4519d1b2415f)

| Metric | What it measures |
| - | - |
| **Requests** and **Avg Requests / min** | How much traffic the deployment served |
| **Error Rate** | The share of requests that failed. Hover to see the count. |
| **Request Latency** | The time from request to complete response, at the 50th, 95th, and 99th percentiles. It includes failed requests. |
| **Time to First Token** | The time until the first token of the response. For a non-streaming request, it equals the full latency. |
| **Token Throughput** | Input and output tokens per second |

## READ THE CHARTS

| What you see | Likely cause | What to do |
| - | - | - |
| Errors rise | Cold starts, rate limits, or prompts that are too long | Open the failed requests on the **Inference Logs** tab and read their messages. See [Errors](https://docs.oumi.ai/guides/deployment/call.md#errors). |
| Time to first token is high on some requests | Cold starts, or requests that wait for a busy replica | Set min replicas to 1 to avoid cold starts, or raise max replicas for concurrent traffic. See [Autoscaling](https://docs.oumi.ai/guides/deployment/deploy-a-model.md#autoscaling). |
| Latency is high, but time to first token is normal | Long responses | Lower `max_tokens`, or stream the response so users see it sooner. |
| Requests drop to zero | No traffic, or an application error before the request reaches Oumi | Check your application. With no traffic, quality monitoring has nothing to score. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.