Language & LLMs

What Is LoRA?

LoRA, short for Low-Rank Adaptation, is a parameter-efficient fine-tuning method that inserts small trainable low-rank matrices into a model while freezing the original weights. This dramatically reduces the number of parameters that must be trained and stored. It allows adapting large models with modest hardware.