Fine-tuning
The process of further training a pre-trained AI model on a specific dataset to specialize its capabilities
What is Fine-tuning?
Fine-tuning is the process of taking a pre-trained model -- one that already understands language in general -- and training it a little more on a focused dataset so it becomes an expert in a particular domain or task.
Imagine hiring a seasoned chef (the pre-trained model) and then having them spend a few weeks in your restaurant learning your specific recipes and plating style. They already know how to cook; fine-tuning teaches them your way of cooking.
How Does It Work?
- Start with a base model -- a large language model that has been pre-trained on broad internet text.
- Prepare a curated dataset -- a collection of examples that represent the task you care about (e.g., medical Q&A pairs, legal document summaries, or customer service dialogues).
- Train further -- the model's parameters are updated on this new data, usually with a lower learning rate so it adapts without forgetting what it already knows.
Fine-tuning vs. Prompt Engineering
Prompt engineering adjusts what you say to the model. Fine-tuning adjusts the model itself. Fine-tuning is more powerful for deep specialization but requires data, compute, and expertise. Prompt engineering is quicker and cheaper for lighter customization.
Key Examples
- Fine-tuning GPT on medical literature to build a clinical assistant.
- Fine-tuning LLaMA on a company's internal documents for enterprise search.