OVERVIEW

The following defines the configuration schema used for training models in Oumi. The configuration includes:

Model loading configuration
Dataset configuration
Training parameters
Logging and evaluation settings
Parameter-efficient fine-tuning (PEFT)

Fields marked with (*) are required.

SCHEMA STRUCTURE

{
  "model": {},
  "data": {},
  "training": {},
  "peft": {}
}

MODEL *

Defines the model and tokenizer configuration.

Field	Type	Required	Description
modelName	string	✓	HuggingFace model name or path
tokenizerName	string		Tokenizer name (defaults to model name)
tokenizerPadToken	string		Override pad token for tokenizer
modelMaxLength	number		Maximum sequence length
torchDtype	string		PyTorch dtype (e.g. `float16`, `bfloat16`)
trustRemoteCode	boolean		Allow execution of custom model repo code
attnImplementation	string		Attention implementation (e.g. `flash_attention_2`)
chatTemplate	string		Custom Jinja chat template
enableLigerKernel	boolean		Enable Liger fused kernel optimizations

CONSTRAINTS

Field	Constraint
modelName	Minimum length: `1`

EXAMPLE

{
  "model": {
    "modelName": "meta-llama/Llama-3-8B",
    "tokenizerName": "meta-llama/Llama-3-8B",
    "torchDtype": "bfloat16",
    "modelMaxLength": 4096
  }
}

DATA *

Defines datasets used for training and evaluation.

Field	Type	Required	Description
train	object	✓	Training dataset configuration
validation	object		Validation dataset configuration
test	object		Test dataset configuration

TRAIN *

Training dataset definition.

Field	Type	Required	Description
datasets	object[]	✓	List of datasets

CONSTRAINTS

Field	Constraint
datasets	Exactly 1 item

DATASET OBJECT

Field	Type	Required	Description
datasetId	integer	✓	Dataset identifier

RANGE

Field	Range
datasetId	(0..9007199254740991]

VALIDATION

Validation dataset configuration.

Field	Type	Description
datasets	object[]	Validation datasets

TEST

Test dataset configuration.

Field	Type	Description
datasets	object[]	Test datasets

TRAINING *

Defines model training parameters.

Field	Type	Description
maxSteps	number	Stop training after this many steps (-1 = unlimited)
numTrainEpochs	integer	Number of full passes over the dataset
learningRate	number	Peak learning rate
lrSchedulerType	enum	Learning rate scheduler
warmupRatio	number	Fraction of warmup steps
warmupSteps	number	Warmup steps (overrides ratio)
optimizer	string	Optimizer name
weightDecay	number	L2 regularization
adamBeta1	number	Adam optimizer beta1
adamBeta2	number	Adam optimizer beta2
maxGradNorm	number	Gradient clipping value
mixedPrecisionDtype	enum	Mixed precision training dtype
enableGradientCheckpointing	boolean	Reduce memory usage
loggingSteps	number	Log every N steps
evalStrategy	enum	Evaluation schedule
evalSteps	number	Eval interval
saveSteps	number	Checkpoint interval
saveFinalModel	boolean	Save model after training
trainerType	enum	Training algorithm
seed	number	Random seed
enableWandb	boolean	Enable Weights & Biases logging
enableTensorboard	boolean	Enable TensorBoard logging
runName	string	Display name for training run

PARAMETER CONSTRAINTS

Field	Range
numTrainEpochs	1–30
learningRate	0.000001–0.001
runName	length 1–128
optimizer	length 1–128

LRSCHEDULERTYPE

Allowed Values
`linear`
`cosine`
`cosine_with_restarts`
`cosine_with_min_lr`
`constant`

MIXEDPRECISIONDTYPE

Allowed Values
`none`
`fp16`
`bf16`

EVALSTRATEGY

Allowed Values
`no`
`steps`
`epoch`

TRAINERTYPE

Allowed Values
`sft`
`opd`

ONPOLICYDISTILLATION

Optional configuration for on-policy distillation.

Requires trainerType = "opd".

PEFT *

Parameter-efficient fine-tuning configuration.

Field	Type	Required	Description
usePeft	boolean	✓	Enable PEFT
peftMethod	string	✓	PEFT method (e.g. `lora`)
loraR	num

Resources

Recipe configs

Recipe examples

MODEL TRAINING RECIPE SCHEMA DEFINITIONS

OVERVIEW

SCHEMA STRUCTURE

MODEL *

CONSTRAINTS

EXAMPLE

DATA *

TRAIN *

CONSTRAINTS

DATASET OBJECT

RANGE

VALIDATION

TEST

TRAINING *

PARAMETER CONSTRAINTS

LRSCHEDULERTYPE

MIXEDPRECISIONDTYPE

EVALSTRATEGY

TRAINERTYPE

ONPOLICYDISTILLATION

PEFT *

​OVERVIEW

​SCHEMA STRUCTURE

​MODEL *

​CONSTRAINTS

​EXAMPLE

​DATA *

​TRAIN *

​CONSTRAINTS

​DATASET OBJECT

​RANGE

​VALIDATION

​TEST

​TRAINING *

​PARAMETER CONSTRAINTS

​LRSCHEDULERTYPE

​MIXEDPRECISIONDTYPE

​EVALSTRATEGY

​TRAINERTYPE

​ONPOLICYDISTILLATION

​PEFT *

OVERVIEW

SCHEMA STRUCTURE

MODEL *

CONSTRAINTS

EXAMPLE

DATA *

TRAIN *

CONSTRAINTS

DATASET OBJECT

RANGE

VALIDATION

TEST

TRAINING *

PARAMETER CONSTRAINTS

LRSCHEDULERTYPE

MIXEDPRECISIONDTYPE

EVALSTRATEGY

TRAINERTYPE

ONPOLICYDISTILLATION

PEFT *