> ## Documentation Index > Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt > Use this file to discover all available pages before exploring further. # QUALITY MONITORING > Score a sample of production traffic with LLM judges and track your deployment's quality over time ## OVERVIEW Quality monitoring uses LLM judges to score a sample of production traffic. The deployment's **Insights** tab plots each evaluator's scores over time, helping you spot regressions before they become widespread. Quality monitoring works on managed deployments with [inference logging](/guides/deployment/inference-logs) enabled. *** ## HOW IT WORKS A monitor attaches to a deployment and holds two things: a set of **judge evaluators** and a **sampling rate**. 1. Your deployment serves a request, and inference logging records it. 2. The monitor samples the percentage of requests you configured. 3. The attached judge evaluators score each sampled request. 4. The **Insights** tab plots a score trend for each evaluator. *** ## SET UP MONITORING There are three ways to start monitoring a deployment. ### WITH THE OUMI AGENT Ask the Oumi Agent to monitor a specific deployment. It checks that inference logging is enabled and proposes judge evaluators and a sampling rate. Review the proposal, then approve it to start scoring. ### DURING DEPLOYMENT The **Deploy Model** dialog includes a **Monitoring** step, right after you configure the deployment. It is optional: add one or more judge evaluators and pick a sampling rate to enable monitoring, or click `Skip` to move on. Anything you set here can be changed later from the deployment's **Info & Configuration** tab. ### FROM THE CONFIGURATION TAB On the deployment detail page, open the **Info & Configuration** tab and scroll to the **Monitoring** section: * **Status** shows whether a monitor is actively scoring this deployment's traffic. * **Evaluators** lists the judges scoring sampled traffic. Click `+ Add evaluator` to attach one; adding the first evaluator creates the monitor with a 10% sampling rate. * **Sampling rate** sets the share of production requests that get scored. The dropdown offers 10%, 20%, 50%, and 100%. Changes apply immediately. *** ## READ THE INSIGHTS TAB The **Insights** tab is where monitoring results land. * The **Time window** selector zooms from 1 hour out to 30 days, with a `Custom` option for an exact range. The tab shows up to 30 days of score history. * A summary pill per evaluator shows its mean score over the visible window, as a percentage. * Each evaluator gets its own **score trend chart**, one point per scored request. If a judge fails to produce scores, a warning badge appears on that evaluator's chart with the failure count and reasons on hover, such as provider quota, authentication, rate limits, or timeouts. These are judge failures, not low model scores; they usually point at the judge's provider key or configuration rather than your model. *** ## INVESTIGATE A DROP When a trend dips, drill into the exact traffic behind it: Click `Select Area to Investigate` and drag across any chart to mark the time range you care about. Click `View Selected Logs` to open the **Scored Inference Logs** table: one row per scored request, with the judgment each evaluator gave it. Expand a row to read the full judge explanations for that request. Click `Run Failure Modes Analysis` to cluster the failing responses in the selected range into recurring patterns. The report opens on the deployment's **Failure Modes** tab when it is ready. A failure modes report groups the reasons that production responses scored poorly, helping you address the highest-impact pattern first. From there, use [compounding intelligence](/guides/deployment/compounding-intelligence) to generate targeted training data, retrain the model, and verify the improvement after redeployment. *** ## MANAGE MONITORED EVALUATORS * **Versions are pinned.** A monitor scores with the exact evaluator version you attached. * **Removing an evaluator** stops scoring against it. You can add it back later. Removing the last monitored evaluator turns off monitoring for the deployment and permanently deletes the monitor along with all of its historical scores. This cannot be undone. *** ## WHAT'S NEXT Turn monitoring insights into targeted training data and an improved model. Watch the operational side: traffic, errors, latency, and throughput.