Knowledge sharing between digital systems running different copies of the same model is extremely efficient because if you have two copies that have seen different data, they can share gradient updates by averaging weights across a trillion parameters, achieving vastly higher knowledge compression than distillation

causalpending

Speaker

Geoffrey Hinton

Evidence Quote

“if you got a trillion weights you're sharing a trillion numbers so it's an incredible bandwidth of sharing”

Source

Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024— Vector Institute
Created: 8/11/2026, 7:34:33 AM

Your notes

Loading notes...