> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oumi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# EVALUATE A MODEL

> Score a model against your own criteria and find where it fails

This example uses the [banking77 validation dataset](https://huggingface.co/datasets/oumi-ai/banking77-oumi-quickstart/resolve/main/banking77_val_basic.jsonl?download=true).

<Steps>
  <Step title="Upload the validation data">
    ```bash theme={null}
    oumi-cli datasets upload ./banking77_val_basic.jsonl --display-name "banking77-val"
    ```

    Note the dataset ID it returns.
  </Step>

  <Step title="Create a classification evaluator">
    ```bash theme={null}
    cat > evaluator.json <<EOF
    {
      "displayName": "banking77-label-match",
      "description": "Checks the predicted intent label against the ground-truth label.",
      "params": {
        "evaluatorType": "classification",
        "labelField": "label",
        "labelExtractionMode": "auto"
      }
    }
    EOF
    ```

    ```bash theme={null}
    oumi-cli evaluators create --input-json-file evaluator.json
    ```

    Note the evaluator ID it returns. Valid labels are auto-discovered from the dataset; run `oumi-cli evaluators generate-template --type classification` to see every option.
  </Step>

  <Step title="Write an evaluation recipe">
    Set the three IDs, then paste the block. `MODEL_ID` is the model you want to score, `DATASET_ID` is the validation set from the upload step, and `EVALUATOR_ID` is the one you just created.

    ```bash theme={null}
    MODEL_ID=1
    DATASET_ID=1
    EVALUATOR_ID=1

    cat > eval-recipe.json <<EOF
    {
      "displayName": "banking77-eval",
      "recipeConfig": {
        "type": "evaluate",
        "evaluationConfig": {
          "evaluationType": "single_model",
          "modelIdentifier": { "modelType": "CUSTOM_CLOUD_STORAGE", "modelId": $MODEL_ID },
          "evaluators": [{ "evaluatorId": $EVALUATOR_ID }],
          "inferenceConfig": {},
          "dataset": { "datasetId": $DATASET_ID },
          "generateFailureModes": true
        }
      }
    }
    EOF
    ```

    To score a base model instead of one you trained, use `{ "modelType": "PRETRAINED", "modelName": "..." }`, or list what is available with `oumi-cli evaluations available-models`.
  </Step>

  <Step title="Create the recipe and run it">
    ```bash theme={null}
    oumi-cli recipes create --type evaluate --input-json-file eval-recipe.json
    ```

    ```bash theme={null}
    oumi-cli evaluations run --name "banking77-eval" --recipe <RECIPE_ID>
    ```
  </Step>
</Steps>

## READING THE RESULTS

```bash theme={null}
oumi-cli evaluations get <EVALUATION_ID>
oumi-cli evaluations results <EVALUATION_ID>
oumi-cli evaluations download <EVALUATION_ID> --out results.jsonl
```

## VIEWING FAILURE MODES

```bash theme={null}
oumi-cli evaluations failure-modes <EVALUATION_ID>
```

<Columns cols={2}>
  <Card title="Train a model" icon="dumbbell" href="/cli/workflows/train-a-model">
    Retrain against the gaps you found.
  </Card>

  <Card title="Evaluations reference" icon="list" href="/cli/commands/evaluations">
    Every flag on every evaluation command.
  </Card>
</Columns>