weight pruning
Technique in neural networks of removing or reducing the importance of certain connections (weights) between neurons to reduce model size and computational cost while maintaining performance
As defined in When Scientists Go Mad: The Unthinkable Thoughts We Can't Think
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “weight pruning” →
- The speaker prepares for interviews by emulating guests over weeks until able to imagine almost any question from their perspective and predict their likely answers—a process analogous to weight pruning and bit-depth reduction in neural networks, creating a low-resolution model of the guest compressed into the speaker's mind.When Scientists Go Mad: The Unthinkable Thoughts We Can't Think
- Language models and neural networks can be viewed as containing low-bit-depth representations of individuals, analogous to how the host's mind contains compressed copies of his interview subjects, and this extends to an ecosystem of 'ghosts' proportional to how many people one has interviewed.When Scientists Go Mad: The Unthinkable Thoughts We Can't Think