Knowledge sharing between digital systems running different copies of the same model is extremely efficient because if you have two copies that have seen different data, they can share gradient updates by averaging weights across a trillion parameters, achieving vastly higher knowledge compression than distillation
causalpending
Speaker
Geoffrey HintonEvidence Quote
“if you got a trillion weights you're sharing a trillion numbers so it's an incredible bandwidth of sharing”
Source
Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024— Vector InstituteCreated: 8/11/2026, 7:34:33 AM
Your notes
Loading notes...