Glossary term
Glossary term
Training and Fine-Tuning
Lightweight trainable modules inserted between frozen pre-trained layers, enabling parameter-efficient task adaptation.
Houlsby et al. (2019) introduced adapters for BERT, adding 3.6% extra parameters per task while matching full fine-tuning performance on GLUE - demonstrating that a single pre-trained model can serve multiple tasks via task-specific adapters.
AdapterHub (HuggingFace) hosts 700+ pre-trained adapter modules for BERT, RoBERTa, and T5 across tasks and languages - developers load a base model and plug in a 5MB adapter instead of fine-tuning all 340M parameters.
LLM serving systems (LoRAX, S-LoRA) serve hundreds of adapter variants from a single base model instance by dynamically swapping adapters per request, enabling cost-efficient multi-tenant personalisation at inference time.