Skip to main content
The base model decides how fast a fine-tuned model responds and what it costs to serve. Its size and pretraining also set a ceiling on what the model can learn. Below that ceiling, fine-tuning on good data with well-tuned settings can improve the model on your task by a large margin. FAiE recommends a base model from your task and your data, such as “Which base model should I fine-tune to route support tickets, with fast responses?” It also tells you which base models support a training method and which ones Oumi can deploy after training.

SIZE AND LATENCY

Smaller models respond faster and cost much less to serve at high volume. Larger models reason and follow instructions better, at a higher latency and cost. Start with a small model, and move to a larger one only if the evaluation shows that the small model can’t reach your quality goal. To narrow the candidates, try a few of your prompts on them in the Model Playground. Look for a base model that understands the task and sometimes completes it, but has plenty of room to improve. If a base model fails badly, the task is likely too complex for it, and a larger base model is a better start.

TASK SCOPE

Small models work well for narrow tasks with a restricted output, such as classification, entity extraction, and routing. Save larger models for open-ended generation, problem solving, and tasks with strict but varied rules for the response format.

DENSE OR MIXTURE-OF-EXPERTS

A dense model uses all its parameters for every token. A mixture-of-experts (MoE) model routes each token to a few expert layers, which gives it more capacity for a lower inference cost. MoE models can be harder to fine-tune, because a small or narrow dataset may not train every expert evenly. If you are new to fine-tuning, or your dataset is small, a dense model gives more predictable results.

REASONING

Most recent base models can reason step by step before they answer, which improves results on complex tasks but adds tokens, latency, and cost.

TOOL USE

If your model calls tools, such as search, retrieval, or your own APIs, choose a base model that was trained for tool use. The Qwen3.5 and later, Gemma 4, and gpt-oss models are trained with extensive tool-use data.

CONTEXT LENGTH

Each base model trains up to a maximum input length, and training truncates longer rows. If your rows are long, such as retrieval with large documents or summarization, choose a base model whose training context fits them, or shorten the rows.

LANGUAGE SUPPORT

Model families support different languages. The Gemma and Qwen families cover many languages, and Qwen is a strong choice for Asian languages, especially Chinese. SmolLM models are strongest in English. Check the model card of your base model for the languages it supports.

DATA PROVENANCE AND LICENSES

All base models are open-weight models with public model cards. Most were trained partly on synthetic data. If training data transparency or license terms matter for your deployment, read the model card before you train.

AVAILABLE MODELS

These are the base models you can fine-tune as of October 2026. The Base Model picker in the training form shows the current list. For the models you can use for inference, evaluation, synthesis, and deployment, see Supported models. Training context is the longest row, in tokens, that supervised fine-tuning trains on. Training truncates longer rows, as described in Context length. Other training methods can train at a different length.

COMPACT MODELS (UNDER 2B PARAMETERS)

MID-SIZE MODELS (2B TO 9B PARAMETERS)

LARGER MODELS (OVER 9B PARAMETERS)