What is transfer learning and why is it fundamental to modern ML applications?

Transfer learning reuses a model trained on one task as the starting point for a different task. Instead of training from scratch (which requires massive data and compute), you fine-tune a pre-trained model on your specific data. Foundation: large models pre-trained on vast datasets (GPT on internet text, BERT on books and Wikipedia, ResNet on ImageNet) learn general representations that transfer to downstream tasks. Types: feature extraction (freeze pre-trained layers, train only a new classification head), fine-tuning (update all or some pre-trained layers with a lower learning rate), and adapter methods (insert small trainable modules while keeping the pre-trained model frozen). Why it works: early layers learn universal features (edges and textures in vision, grammar and semantics in NLP), and these transfer across tasks. Later layers learn task-specific features that need adaptation. Practice: for NLP, start with a pre-trained transformer (BERT, RoBERTa, DeBERTa), add a task-specific head (classification layer, token classification layer, sequence-to-sequence head), and fine-tune on your labeled data (typically 1000-10000 examples). For vision, start with a pre-trained CNN or ViT, replace the final classification layer, and fine-tune with your images (often effective with as few as 100 examples per class). Benefits: dramatically reduces data requirements (from millions to hundreds of examples), reduces training time (from weeks to hours), and often achieves better performance than training from scratch. Modern impact: transfer learning enabled the current AI revolution because it makes powerful models accessible without requiring massive training resources.