SIMULATED USERS

A simulated user is a model that plays the user in a conversation. After the assistant answers, the simulated user writes the next user message, so a single dataset row becomes a conversation with several user turns. You steer how it behaves on each row with a user persona. You add a simulated user by choosing a User Model on a synthesis or evaluation job. See Generate data with tools and a simulated user and Evaluate with tools and a simulated user for the steps.

A simulated user takes turns only when the job also attaches at least one tool implementation. A job with a user model and no tool implementations generates one assistant response to the first user message.

HOW A CONVERSATION PROCEEDS

Each row starts from its first user message. Oumi keeps everything up to and including that message, removes every later message, and then runs this loop:

The assistant model responds. If the response contains tool calls, each call goes to the tool implementation attached for that tool, and the result is appended as a tool message. The assistant responds again, and the loop repeats until the assistant replies without a tool call.
The simulated user reads the conversation and writes the next user message. The assistant then responds again, starting from step 1.
The conversation ends when the simulated user decides it is finished, or when it reaches the maximum number of user turns.

Without a user model, each conversation has exactly one user turn, and the assistant may still make several tool calls within it. The simulated user follows a platform-defined instruction. It writes brief messages in the user’s voice, never answers its own request, and builds on what was already said. When its goals are met, or when the assistant clearly cannot help further, it ends the conversation. You do not see an end marker in the output. The conversation simply stops after the last assistant response. The simulated user also answers calls to interactive tool implementations, so a job that attaches one must set a user model.

TURN LIMITS

Limit	Value
User turns per conversation	1 to 20, set by Max user turns (`maxUserTurns`). Includes the first user message. Defaults to 20 when unset.
Consecutive assistant turns	20
Assistant turns per user turn	40

A call to an interactive tool implementation is answered by the simulated user, so it resets the consecutive count. The per-user-turn limit still applies. Set Max user turns to a small value, such as 5, unless your real conversations run longer. If you have real traces, count the user turns in them and use the maximum plus a small buffer.

STEER THE USER WITH A PERSONA

Store a persona in each row’s metadata under the reserved key user_persona. Oumi adds it to the simulated user’s instruction for every turn it writes. A dataset for a generate completions job or an evaluation looks like this, one conversation per line:

{"messages": [{"role": "user", "content": "Hi, my order hasn't arrived yet. Can you check on it?"}], "metadata": {"user_persona": "You are a busy parent who ordered a stroller 10 days ago. Your order id is A-48213, but you only give it when asked. You want a delivery date, and you get impatient if the assistant asks the same question twice."}}
{"messages": [{"role": "user", "content": "I want to change the shipping address on something I bought."}], "metadata": {"user_persona": "You are a first-time customer who is unsure how online orders work. You moved last week. You have the order confirmation email open and read details from it when asked."}}

Datasets made with Import observability logs sometimes carry a user_persona already. You can read a row’s persona in the dataset viewer. Open a row, and the persona appears in the User persona section.

GENERATE PERSONAS WITH SYNTHESIS

In a general synthesis recipe, create an attribute with the id user_persona. Oumi copies its value into each conversation’s metadata. Build it from separate sampled attributes, so that each dimension of the persona varies independently:

{
  "sampledAttributes": [
    {
      "id": "goal",
      "name": "Goal",
      "description": "What the customer wants",
      "possibleValues": [
        {"id": "late", "name": "Late order", "description": "You want to know why your order is late", "sampleRate": 0.5},
        {"id": "address", "name": "Address change", "description": "You want to change the shipping address on an order", "sampleRate": 0.5}
      ]
    },
    {
      "id": "temperament",
      "name": "Temperament",
      "description": "How the customer behaves",
      "possibleValues": [
        {"id": "patient", "name": "Patient", "description": "patient and polite", "sampleRate": 0.5},
        {"id": "rushed", "name": "Rushed", "description": "in a hurry and terse", "sampleRate": 0.5}
      ]
    }
  ],
  "transformedAttributes": [
    {
      "id": "user_persona",
      "transformationStrategy": {
        "type": "string",
        "stringTransform": "You are a customer who is {temperament.description}. {goal.description}. Share your order id only when asked."
      }
    }
  ]
}

These attributes go in strategyParams of a general synthesis recipe. See Synthesis recipes for a complete request.

WRITE GOOD PERSONAS

Vary personas across rows. Similar personas produce similar conversations. Vary the user’s goal, background or expertise, temperament, and how clearly they state the request.
Write the goal from the user’s side. Describe what the user wants (“You want to cancel a hotel reservation”), not what the assistant should do.
Cover every tool. Include goals that require your less common tools. Do not name tools in the persona.
Withhold some details. Give the persona specifics, such as an order id or a budget, that it reveals only when asked. This makes the assistant ask clarifying questions.
Do not over-script the user. A clear goal usually works better than a detailed script.

REQUIREMENTS

The user model must be a hosted API model. Models that run on a GPU deployment cannot play the user.
Max user turns (maxUserTurns) requires a user model.
For the full field reference, see agenticInferenceConfig.

Getting started

Oumi workflow

SIMULATED USERS

HOW A CONVERSATION PROCEEDS

TURN LIMITS

STEER THE USER WITH A PERSONA

GENERATE PERSONAS WITH SYNTHESIS

WRITE GOOD PERSONAS

REQUIREMENTS

​HOW A CONVERSATION PROCEEDS

​TURN LIMITS

​STEER THE USER WITH A PERSONA

​GENERATE PERSONAS WITH SYNTHESIS

​WRITE GOOD PERSONAS

​REQUIREMENTS

HOW A CONVERSATION PROCEEDS

TURN LIMITS

STEER THE USER WITH A PERSONA

GENERATE PERSONAS WITH SYNTHESIS

WRITE GOOD PERSONAS

REQUIREMENTS