> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# SYNTHESIZE WITH A HARNESS

> Generate conversation datasets with your own agent harness in the Builder

Use Bring Your Own Harness (BYOH) to generate conversation datasets with your existing agent. Your harness runs in your environment and keeps its prompts, tools, context management, memory, and application logic.

When you generate a dataset from scratch, choose **Your harness** in the Builder. Oumi generates the opening messages for each conversation, your harness produces the assistant responses, and Oumi saves the full conversations to a new dataset.

***

## BEFORE YOU START

You need:

* A [harness registered in your project](https://docs.oumi.ai/guides/harnesses/register.md).
* An API-hosted model for your agent to call during the run.
* A [synthesis recipe](https://docs.oumi.ai/guides/synthesis/recipes.md) that produces conversation openings ending with a user message.

> [!NOTE]
>
> Harness synthesis is available only through **Generate from Scratch**. **Generate Completions** and **Generate from Failure Modes** do not support harnesses.

***

## HOW A HARNESS RUN WORKS

A synthesis run with a harness has two stages:

1. **Oumi generates the conversation openings.** The model you select under **Model & Inference** generates the recipe's attributes, such as a question, a persona, or a scenario. A chat transform assembles them into the opening messages of each conversation.
2. **Your harness completes the conversations.** Oumi sends each opening to your harness with a connection to the same model. Your harness runs the agent and its tools, then returns the resulting messages. Oumi saves the full conversation to the output dataset.

The messages produced by the chat transform must end with a user message so your harness can generate the next assistant response. Include a system prompt if your recipe uses one. If an opening ends with an assistant message, that row fails.

***

## GENERATE A DATASET

1. Go to **Datasets** and click **Create Dataset**, then choose **Generate from Scratch** to open the Builder.
2. Under **Model & Inference**, select the API-hosted model your agent should call.
3. Under **Agent Runtime**, select **Your harness**. In the **Harness** section that appears, choose a registered endpoint.

Your harness runs its own tools. Selecting **Your harness** hides the Builder's **Tools** section, and Oumi's tool implementations are not used during the run.

Configure the rest of the recipe as you normally would:

| Input | What to select |
| - | - |
| **User Model** | A model to simulate user replies if your agent asks follow-up questions. See below. |
| **Input Sources** and **Sampled Attributes** | The seed data and variations that shape each opening. |
| **Generated Attributes** | The content the synthesis model writes for each row. |
| **Transformed Attributes** | The chat transform that assembles the opening messages. The last message must be from the user. |
| **Output Configuration** | The number of samples and the split between train, validation, and test. |

Review the configuration and click **Execute**. Give the job a **Display Name** and click **Generate Dataset**.

You can follow the run on the [Jobs](https://docs.oumi.ai/guides/jobs.md) page. When the job completes, the new dataset appears on your **Datasets** page.

***

## ADD A SIMULATED USER

If your agent asks questions during a conversation, expand **User Model** in the Builder and select a model to answer them. This setting is separate from the model selected under **Model & Inference**.

When your agent pauses for input, Oumi generates the user's reply and sends it back so the conversation can continue. If your recipe synthesizes a user persona, Oumi uses it to guide those replies. The persona is not shared with your agent.

You can leave **User Model** empty if your agent does not need follow-up input. If it requests input without a user model configured, that row fails.

***

## BILLING

Synthesis with a harness follows the same billing model as other synthesis runs:

* **Attribute generation:** Token usage to generate the conversation openings.
* **Harness turns:** Token usage for the calls your harness makes through the model connection.
* **Simulated user:** Token usage to generate user replies, if a user model is configured.

Oumi does not bill for your agent's infrastructure or for model calls made outside the provided connection.

***

## TROUBLESHOOTING

| Issue | What to check |
| - | - |
| **Agent Runtime** does not appear in the Builder. | Confirm that at least one harness is registered in the same project. |
| Your registered endpoint is missing from the **Harness** selector. | Confirm that you registered it in the same project as the synthesis run. |
| Oumi cannot connect to the endpoint. | Check the URL on the **Harnesses** page and confirm with your team that the service is running and publicly reachable. |
| The endpoint rejects the run. | Check its authentication settings and update the credential if needed. |
| Rows fail when the agent asks a question. | Add a **User Model** in the Builder so Oumi can generate replies. |
| Rows fail before your harness responds. | Confirm that the chat transform in **Transformed Attributes** ends with a user message. |
| Some conversations fail to complete. | Ask your team to check the endpoint logs for errors or timeouts. |
| No rows complete. | Check the recipe, connection settings, and endpoint logs before starting another synthesis run. |

For endpoint implementation details, see the [harness protocol](https://docs.oumi.ai/guides/harnesses/protocol.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.