Overview
The following defines the configuration schema used for training models in Oumi.
The configuration includes:
- Model loading configuration
- Dataset configuration
- Training parameters
- Logging and evaluation settings
- Parameter-efficient fine-tuning (PEFT)
Fields marked with (*) are required.
Schema structure
{
"model": {},
"data": {},
"training": {},
"peft": {}
}
model *
Defines the model and tokenizer configuration.
| Field | Type | Required | Description |
|---|
| modelName | string | ✓ | HuggingFace model name or path |
| tokenizerName | string | | Tokenizer name (defaults to model name) |
| tokenizerPadToken | string | | Override pad token for tokenizer |
| modelMaxLength | number | | Maximum sequence length |
| torchDtype | string | | PyTorch dtype (e.g. float16, bfloat16) |
| trustRemoteCode | boolean | | Allow execution of custom model repo code |
| attnImplementation | string | | Attention implementation (e.g. flash_attention_2) |
| chatTemplate | string | | Custom Jinja chat template |
| enableLigerKernel | boolean | | Enable Liger fused kernel optimizations |
Constraints
| Field | Constraint |
|---|
| modelName | Minimum length: 1 |
Example
{
"model": {
"modelName": "meta-llama/Llama-3-8B",
"tokenizerName": "meta-llama/Llama-3-8B",
"torchDtype": "bfloat16",
"modelMaxLength": 4096
}
}
data *
Defines datasets used for training and evaluation.
| Field | Type | Required | Description |
|---|
| train | object | ✓ | Training dataset configuration |
| validation | object | | Validation dataset configuration |
| test | object | | Test dataset configuration |
train *
Training dataset definition.
| Field | Type | Required | Description |
|---|
| datasets | object[] | ✓ | List of datasets |
Constraints
| Field | Constraint |
|---|
| datasets | Exactly 1 item |
Dataset object
| Field | Type | Required | Description |
|---|
| datasetId | integer | ✓ | Dataset identifier |
Range
| Field | Range |
|---|
| datasetId | (0..9007199254740991] |
validation
Validation dataset configuration.
| Field | Type | Description |
|---|
| datasets | object[] | Validation datasets |
test
Test dataset configuration.
| Field | Type | Description |
|---|
| datasets | object[] | Test datasets |
training *
Defines model training parameters.
| Field | Type | Description |
|---|
| maxSteps | number | Stop training after this many steps (-1 = unlimited) |
| numTrainEpochs | integer | Number of full passes over the dataset |
| learningRate | number | Peak learning rate |
| lrSchedulerType | enum | Learning rate scheduler |
| warmupRatio | number | Fraction of warmup steps |
| warmupSteps | number | Warmup steps (overrides ratio) |
| optimizer | string | Optimizer name |
| weightDecay | number | L2 regularization |
| adamBeta1 | number | Adam optimizer beta1 |
| adamBeta2 | number | Adam optimizer beta2 |
| maxGradNorm | number | Gradient clipping value |
| mixedPrecisionDtype | enum | Mixed precision training dtype |
| enableGradientCheckpointing | boolean | Reduce memory usage |
| loggingSteps | number | Log every N steps |
| evalStrategy | enum | Evaluation schedule |
| evalSteps | number | Eval interval |
| saveSteps | number | Checkpoint interval |
| saveFinalModel | boolean | Save model after training |
| trainerType | enum | Training algorithm |
| seed | number | Random seed |
| enableWandb | boolean | Enable Weights & Biases logging |
| enableTensorboard | boolean | Enable TensorBoard logging |
| runName | string | Display name for training run |
Parameter constraints
| Field | Range |
|---|
| numTrainEpochs | 1–30 |
| learningRate | 0.000001–0.001 |
| runName | length 1–128 |
| optimizer | length 1–128 |
lrSchedulerType
| Allowed Values |
|---|
linear |
cosine |
cosine_with_restarts |
cosine_with_min_lr |
constant |
mixedPrecisionDtype
| Allowed Values |
|---|
none |
fp16 |
bf16 |
evalStrategy
| Allowed Values |
|---|
no |
steps |
epoch |
trainerType
onPolicyDistillation
Optional configuration for on-policy distillation.
Requires trainerType = "opd".
peft *
Parameter-efficient fine-tuning configuration.
| Field | Type | Required | Description |
|---|
| usePeft | boolean | ✓ | Enable PEFT |
| peftMethod | string | ✓ | PEFT method (e.g. lora) |
| loraR | num | | |
