Glossary term
Glossary term
Training and Fine-Tuning
Phenomenon where a neural network loses previously learned knowledge when trained on new tasks or data.
Early instruction-tuning of Llama 2 on customer-service data caused catastrophic forgetting of coding skills - addressed by mixing general instruction data with domain-specific data in SFT training to preserve breadth.
Elastic Weight Consolidation (EWC, DeepMind) reduces catastrophic forgetting by regularising important weights - used in continual RL experiments to prevent a game-playing agent from forgetting earlier game skills.
LoRA adapters partially mitigate catastrophic forgetting by freezing the base model - only the small adapter weights are updated during fine-tuning, leaving the base model's knowledge intact for general use.