Fine-Tuning

Artificial Intelligence / 11 Sep 2026 / 9 min read

Adapt Language Models with Prefix Tuning

Adapt Language Models with Prefix Tuning Full fine-tuning changes a model’s weights for each task. That can be effective, but storing and serving a separate full checkpoint for every task becomes expensive as model size and task count grow. Prefix tuning offers a different arrangement: keep the pretrained model frozen and train a small set of task-specific states that participate in attention. This article builds a practical mental model for prefix tuning, shows how it differs from text prompts and low-rank weight adapters, and explains the trade-offs that matter when training or serving several task variants.

Artificial Intelligence LLM

Artificial Intelligence / 10 Sep 2026 / 8 min read

Mask Prompt Tokens During Instruction Fine-Tuning

A supervised language-model example often contains more than the text you want the model to produce. It may include a system message, a user request, separators, and an assistant answer. If you compute next-token loss over the entire sequence, the model is trained to predict all of those tokens, not just the assistant response. That may be intentional for some training objectives. For instruction fine-tuning, though, developers often want the prompt to provide context while only selected response tokens contribute to the supervised loss. A loss mask makes that distinction explicit.

Artificial Intelligence LLM

Artificial Intelligence / 10 Sep 2026 / 11 min read

Improve LLM Fine-Tuning with Rejection Sampling

Improve LLM Fine-Tuning with Rejection Sampling Suppose you can tell a good model response from a bad one, but writing thousands of ideal responses by hand is expensive. A capable language model may already produce acceptable answers some of the time. The problem is that those answers are mixed with weaker ones. Rejection sampling fine-tuning turns that observation into a data-generation loop. For each prompt, generate several candidate responses, evaluate them, keep responses that satisfy a selection rule, and use the accepted prompt-response pairs for supervised fine-tuning. The method can concentrate training on behavior you want without requiring a human to author every target from scratch.

Artificial Intelligence Large Language Models

Artificial Intelligence / 08 Sep 2026 / 10 min read

Model Soups for Combining Fine-Tuned Models

A hyperparameter sweep often leaves you with several fine-tuned models that are individually useful. The usual workflow keeps the checkpoint with the best validation score and discards the rest. An ensemble can use several checkpoints, but then every request may require multiple model evaluations, increasing inference cost and operational complexity. A model soup offers a third option: average the parameters of compatible fine-tuned models and deploy the resulting parameter set as one model. The technique is simple, but its simplicity can be misleading. Parameter averaging is meaningful only when the checkpoints are sufficiently compatible, and the averaged model still needs independent evaluation.

Artificial Intelligence Machine Learning

Artificial Intelligence / 05 Sep 2026 / 9 min read

Direct Preference Optimization for Language Models

A language model can learn to imitate examples with supervised fine-tuning, but imitation alone does not directly express a common requirement: for the same prompt, one acceptable response may be preferable to another. Preference data represents that requirement as comparisons. A training record contains a prompt, a chosen response, and a rejected response. Direct preference optimization (DPO) uses those pairs to adjust a language model so that the chosen response becomes more favored relative to the rejected one, while comparing the update with a fixed reference model.

Artificial Intelligence Language Models

Artificial Intelligence / 03 Sep 2026 / 10 min read

LoRA for Parameter-Efficient Fine-Tuning

Fine-tuning a large model does not always require updating every model parameter. Low-Rank Adaptation (LoRA) takes advantage of this idea by keeping the original model weights frozen and learning much smaller matrices that modify selected layers. For developers, LoRA changes what must be trained, stored, and moved between experiments—not just the size of the fine-tuning job. That distinction determines when it is useful, what it does not save, and how adapter choices affect model behavior.

Artificial Intelligence LLM

Artificial Intelligence / 03 Sep 2026 / 6 min read

Choose Between Prompting, RAG, and Fine-Tuning

When an AI application produces weak results, teams often jump directly to fine-tuning. That can be the right choice, but many problems are cheaper and easier to solve with better prompting or retrieval-augmented generation (RAG). The three approaches change different parts of the system. Prompting changes the instructions and context given at inference time. RAG supplies relevant external information at inference time. Fine-tuning changes the model’s learned parameters through additional training.

Artificial Intelligence LLM