Skip to main content
Oumi rebuilds each conversation in a trace export, with its system prompt, turns, tool definitions, and tool calls. Import traces when you want training or evaluation data that reflects how real users talk to your agent. Real traces carry the tool calls, system prompts, and multi-turn structure your agent actually produced, which synthetic data only approximates. Use synthesis instead when you have no traffic yet, or need cases your users have not hit.

TRACES OUMI CAN READ

Oumi rebuilds a conversation from the model calls in a trace. It reads model calls logged in the OpenAI chat-completions shape:
  • Langfuse: A GENERATION observation whose input and output hold the request and the response as JSON strings.
  • LangSmith: An llm run whose inputs.messages holds OpenAI or LangChain messages.
Traces in other shapes, such as another provider’s message format or steps that aren’t model calls, can import with missing content or be skipped. Import a small part of your traces first to confirm that Oumi reads them correctly. Oumi reads raw trace exports in JSONL, one record per line. A raw trace export keeps the links between the steps of a trace, which Oumi needs to rebuild the conversation. Langfuse’s JSON export and LangSmith’s dataset export drop those links, so Oumi rejects them.

EXPORT FROM LANGFUSE

In Langfuse, open the project’s Tracing tab, and filter to the traces you want. Open Export, and choose As JSONL. See Langfuse’s Export from UI guide. Langfuse Export menu with the As JSONL option Each line is one observation, which is a single step that Langfuse recorded, such as a span, a generation, or a tool call. Observations from the same trace share a traceId, and Oumi groups them into one conversation. Oumi builds the turns from the GENERATION lines and keeps the other lines as metadata. This example is one trace with a span and a generation whose response is a tool call:

EXPORT FROM LANGSMITH

The LangSmith UI has no export for raw traces, so you export them with the LangSmith SDK. You need a LangSmith API key with read access to the project, which you create in LangSmith under Settings > API Keys. Install the SDK, and set the key in your environment:
Then run a script like this one. To export only part of the project, add run filters or the start_time and end_time parameters to list_runs. A large project exports faster in date ranges.
Each line is one run, such as a chain, llm, or tool run. Runs from the same trace share a trace_id, and Oumi groups them into one conversation. Oumi picks the llm run with the most messages and rebuilds the conversation from its inputs.messages and outputs. Tool results already appear in those messages as tool turns. For session grouping, Oumi reads extra.metadata.session_id, because the top-level session_id is the ID of the LangSmith project. This example is one trace with a chain run and an llm run:

IMPORT THE FILE

You can import a trace export in two ways:
  • Upload it like any other file. Oumi recognizes Langfuse and LangSmith exports, and it always removes duplicate traces. See Upload datasets.
  • Use Import Observability Logs. Choose it under Create Dataset when you want to name the dataset, state the platform yourself, or keep duplicate traces.
The import runs in the background. The new dataset appears on the Datasets page when it finishes.

DUPLICATE TRACES

Many agents log a trace after every turn. A three-turn conversation then appears as three traces, and each trace repeats the turns before it. Without deduplication, the dataset would contain the same conversation three times at different lengths. Deduplication finds traces whose turns begin another, longer trace, and keeps only the longest one. It is on by default. Turn it off in Import Observability Logs only when you want the partial conversations as separate examples. Deduplication compares the traces within one import, so importing overlapping traces twice creates two datasets with the same conversations.

WHAT THE DATASET CONTAINS

Each row of the new Conversation dataset is one trace that Oumi could read. The row holds the system, user, assistant, and tool turns in trace order, with each turn’s tool_calls and tool_call_id, and the trace’s tools array when it has one. Each row also keeps these metadata fields: source_raw keeps every field Oumi doesn’t use, such as timings, costs, model names, and user IDs. Remove sensitive data from your traces before you export them, because the dataset keeps everything in the export. The dataset works like any other Conversation dataset. You can train on it, evaluate a model against it, or use it as input to synthesis.

LIMITATIONS

  • Oumi imports only Langfuse and LangSmith traces.
  • Each import reads one platform’s export. To combine Langfuse and LangSmith traces, import them separately and combine the datasets.
  • Each import creates a new dataset. It can’t add rows to an existing dataset.
  • Oumi skips conversations that are too long, and imports the rest of the file.

TROUBLESHOOTING

Importing only reads your export file. It doesn’t change your traces in Langfuse or LangSmith.