Glossary term
Glossary term
Agentic Systems
A parameter-efficient technique for fine tuning that "freezes" the model's pre-trained weights (such that they can no longer be modified) and then inserts a small set of trainable weights into the model. This set of trainable weights (also known as "update matrixes") is considerably smaller than the base model and is therefore much faster to train.
LoRA provides the following benefits:
Improves the quality of a model's predictions for the domain where the fine tuning is applied.
Fine-tunes faster than techniques that require fine-tuning all of a model's parameters.
Reduces the computational cost of inference by enabling concurrent serving of multiple specialized models sharing the same base model.
Click the icon to learn more about update matrixes in LoRA.
Created for this library
An enterprise legal team uses Low-Rank Adaptability to adapt a base LLM to its contract corpus with a small number of trainable parameters.
A SaaS team uses LoRA to maintain hundreds of customer-specific adapters on top of one shared base model.
A research team uses LoRA to test many domain adaptations of a foundation model quickly without retraining the base.
Definition source: Google for Developers Machine Learning Glossary | Creative Commons Attribution 4.0 License