factual
Models Start From Random Weights but Structured Architecture
Neural networks are randomly initialized (weights start as random numbers), so in one sense they are a blank slate, but the architecture itself is not random—modern architectures embed substantial structure, or 'inductive biases,' that guide learning, so in another sense they are not a blank slate.
factualpending
Speaker
Raphael MilliereEvidence Quote
“you can think of this structure as embedding some priors, some biases, what's known in computer science as inductive biases that help learning in a specific domain.”
Created: 6/13/2026, 12:19:51 AM
Your notes
Loading notes...