> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# COMPOUNDING INTELLIGENCE
> Turn monitoring insights into targeted training data, retrain your model, and improve it with every production cycle
## OVERVIEW
Compounding intelligence turns production feedback into better models. Quality monitoring reveals where a deployed model struggles, failure mode analysis groups recurring problems, and dataset generation creates targeted examples for retraining.
This guide walks through the workflow in the UI. You can also ask the Oumi Agent to guide you through the same process, from analyzing your traffic on the **Insights** tab to a retrained model ready to redeploy.
Before you begin, enable [quality monitoring](/guides/deployment/monitoring) on a deployment and generate a failure modes report from its production traffic.
***
## THE LOOP
1. **Monitor.** Judge evaluators score a sample of production traffic and plot trends on the **Insights** tab.
2. **Diagnose.** Analyze the traffic behind a score drop to identify recurring failure modes.
3. **Generate.** Create synthetic training data that targets the failure modes you select.
4. **Retrain.** Train a new model on the generated data and your original training set.
5. **Redeploy and verify.** Deploy the new model and use monitoring to confirm the improvement.
***
## STEP 1: CREATE A FAILURE MODES REPORT FROM INSIGHTS
Open the deployment's **Insights** tab. Each judge evaluator has its own score trend:
Click `Select Area to Investigate` and drag across the score drop you want to examine.
Click `View Selected Logs` to confirm that the selected traffic contains the behavior you want to fix. Expand any row to read the judge explanations.
Click `Run Failure Modes Analysis`. Oumi groups the failing responses into recurring patterns and opens the completed report on the deployment's **Failure Modes** tab.
The report helps you address recurring causes instead of fixing individual responses. See [quality monitoring](/guides/deployment/monitoring#investigate-a-drop) for the full investigation workflow.
***
## STEP 2: GENERATE A DATASET FROM THE FAILURE MODES
On the **Failure Modes** tab, select the failure modes you want to address and the specific data items within them. Click `Generate Dataset` to open a confirmation dialog with three settings:
| Field | What it controls |
| -------------------- | --------------------------------------------------------------------------------------------------- |
| **Model** | The hosted model that generates the improved samples. Oumi selects a default when one is available. |
| **Dataset name** | The name of the generated dataset. Leave it blank to let Oumi choose a name. |
| **Samples per item** | How many improved samples to generate for each selected data item (1–20; default: 1). |
Click `Generate Dataset` to start generation. Oumi opens the **Datasets** page, where the new dataset appears when the job finishes.
### ADVANCED MODE
For more control, click `Advanced…`. The Builder opens with your selection preloaded and provides additional settings:
| Section | What it controls |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Failure Mode Selection** | The failure modes and data items used for generation. Your original selection is already loaded. |
| **Inference Configuration** | The generation model, temperature, maximum tokens, seed, and optional requests-per-minute limit. |
| **Dataset Configuration** | The destination dataset and its train, validation, and test split. Choose an existing dataset to append the generated data, or leave the destination empty to create one. Split values must total 1.0. |
To edit the raw synthesis configuration, click `Edit as JSON`, make your changes, and click `Save` to return them to the form. Click `Run Synthesis`, name the dataset, and start the job.
***
## STEP 3: RETRAIN WITH THE NEW DATASET
Train the next version of your model on the original and generated datasets. Using both preserves the original examples while adding targeted coverage for the newly identified failures:
1. On the **Models** page, click `Train New Model` and select `Supervised Fine-Tuning`.
2. Select your base model.
3. In the training dataset selector, choose both the generated dataset and the original training set.
4. Launch the job. See [running training](/guides/training/launch-jobs) for the full configuration options.
When training finishes, [deploy the new model](/guides/deployment/deploy-a-model) and direct traffic to it. Continue monitoring to verify that the targeted scores improve. If scores drop again, repeat the loop with the new traffic.
***
## WHAT'S NEXT
Configure judge evaluators and collect the score trends that start this loop.
Learn how reports group recurring failures in evaluations and production traffic.
Configure and launch a fine-tuning job with your improved training data.
Serve the retrained model and direct production traffic to it.