Skip to main content
An agent conversation is a conversation in which the model calls your tools, reads their results, and keeps talking with a simulated user over several turns. Generate them to train a model that calls tools, or to build a dataset that evaluates one. You add tools and a simulated user to a Generate Completions or Generate Synthetic Data from Scratch job. To generate data from the failures of an evaluation that used tools, see Conversations that call tools.

​
BEFORE YOU START

You need:
  • At least one tool with a tool implementation that answers its calls. The Tools section appears only when your project has at least one tool.
  • A model that supports tool calling. The GET /v1/projects/{project_id}/synthesis:supported_models response reports supportsToolCalling for each model, and oumi-cli datasets supported-models lists the same catalog.
  • For a simulated user, a hosted API model to play the user.

​
GENERATE AGENT CONVERSATIONS

To add tools and a simulated user to a job:
  1. On the Datasets page, click Create Dataset.
  2. Click Generate Completions to add responses to an existing dataset, or Generate Synthetic Data from Scratch to synthesize new conversations.
  3. Configure the model and inputs as usual. The model you select writes the assistant turns.
  4. In the Tools section, open Select tools and implementations and check one implementation for each tool the model may call. You can pick only one implementation per tool name.
  5. Optional: in the User Model section, select a model to play the user, and set Max user turns. The count includes the first user message.
  6. Click Execute, enter a name in Name your new dataset, and click Generate Completions or Generate Dataset.
The simulated user takes turns only when the job also attaches at least one tool implementation. A user model without tool implementations produces single-turn output.
For each row, Oumi keeps the conversation up to and including the first user message and regenerates the rest. Existing assistant responses, tool results, and later user messages are not reused. For how the conversation proceeds and how to steer the simulated user with a persona, see Simulated users.

​
REVIEW THE OUTPUT

Each output row contains the full conversation: the tools the model could call, assistant messages with structured tool_calls, tool results, and any simulated user messages. A tool-calling assistant message often has empty text content because the call is in tool_calls. The dataset’s Overview tab lists, under Tools, the tools that the generating job made available. When a conversation reaches the assistant turn limit, Oumi replaces the last assistant message with a fixed note and handles the row as follows:
  • Generate Synthetic Data from Scratch drops the row. The job completes with a partial failure that lists the row with the reason “The assistant exceeded the maximum tool-call turns.”
  • Generate Completions keeps the row and sets tool_call_limit_exceeded to "true" in its metadata. Filter these rows out before you train on the dataset.

​
SPLIT THE WORK INTO TWO JOBS

Generating conversations with tools and a simulated user costs far more than generating prompts, so split the work:
  1. Run a synthesis job without tools that produces the opening user messages and a user_persona for each row.
  2. Review a sample of rows. Check for duplicates, confirm that personas vary, and confirm that the prompts exercise each tool.
  3. Run Generate Completions on that dataset with tools and a user model, using a strong model as the assistant.
  4. Evaluate the output and keep only rows that pass your evaluators. Also remove rows with repeated tool calls or the tool_call_limit_exceeded flag.

​
WITH THE API

Set agenticInferenceConfig on a POST /synthesis:generate-responses request, or in the recipeConfig of a POST /synthesis:run request. For its fields, see agenticInferenceConfig. Calls use your API key in the X-API-Key header, and both endpoints require a valid payment card on the account.
synthesis:run requires a hosted model, and Document Q&A synthesis does not accept tool implementations. synthesis:generate-responses also accepts custom and pretrained models that support tool calling.