Skip to main content
Inference logs record the requests your deployment serves and the responses it returns. They show what users actually ask your model, which no test set fully predicts. Use them to debug a failed request, and export them as a dataset to evaluate or retrain the model on real traffic.

LOGGING AND RETENTION

Inference logging is on by default for every deployment, including external deployments. To change it, use the Inference Logging toggle when you create the deployment, or the Inference logging section of the deployment’s Info & Configuration tab. Turn logging off when requests contain data you must not store. Serving metrics and quality monitoring are computed from the logs, so they stop when logging is off. Requests served while logging is off are never recorded, so turning it back on doesn’t recover them. Logs are kept for your plan’s retention period. Older requests are no longer available to read, export, or score.

READ THE LOGS

The Inference Logs tab lists one row per request, with its status, a preview of the input and output, the token count, the latency, and the time to first token. Click a row to see the full input and output, the tools the request offered, and the error message if the request failed. Inference Logs tab with one row per request and its status, input, output, tokens, and latency To debug a failure, open the failed request and read its error message next to the input that caused it. For the common errors and their fixes, see Errors.

EXPORT LOGS TO A DATASET

Exported logs become a Conversation dataset of user and assistant turns. Use it as a test set that reflects real traffic, as input to synthesis, or as training data.
  1. On the Inference Logs tab, click Export as Dataset.
  2. Choose Create new dataset and enter a Dataset Name, or choose Append to existing and select a Dataset.
  3. Optionally, set a Start Date and End Date. Leave both empty to include all logs.
  4. Click Generate Dataset.
Only requests with both a prompt and a response are exported. The export runs in the background, and the dataset appears on the Datasets page when it finishes. Raw logs record what the model answered, including its mistakes. To train on corrected answers to the requests the model got wrong, generate data from their failure modes instead. See Continuous improvement.