low rank adapters
Small add-on parameter modules that personalize a model without modifying the full set of base weights.
As defined in Dwarkesh Patel Podcast - 8 Predictions for the Era of Continual Learning
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “low rank adapters” →
- Continual learning may create inference economies of scale because per-company information requires full weight updates rather than low-rank adapters; a sparse model like DeepSeek V3 may need an optimal inference batch size above 2,400 sequences, so individual users serving themselves at batch size 1 could suffer a 100x+ compute efficiency penalty, favoring big organizations.Dwarkesh Patel Podcast - 8 Predictions for the Era of Continual Learning
- Updating one user's weights is different from pooling many weight forks back into the main model; the latter may be technically harder but Patel expects it will eventually be solved.Dwarkesh Patel Podcast - 8 Predictions for the Era of Continual Learning