Low-Rank Adaptation (LoRA) is a fine-tuning technique that freezes all pre-trained model weights and adds trainable rank decomposition matrices into each layer of the Transformer architecture, reducing the number of trainable parameters during fine-tuning by about 10,000 times as well as GPU requirements.

factualpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“low rank adaptation or Laura is a fine tuning technique that freezes all the pre-trained model weights and adds trainable rank decomposition matrices into each layer of the Transformer architecture this greatly reduces the number of trainable parameters during fine tuning by about 10 000 times”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...