factual

Models Start From Random Weights but Structured Architecture

Neural networks are randomly initialized (weights start as random numbers), so in one sense they are a blank slate, but the architecture itself is not random—modern architectures embed substantial structure, or 'inductive biases,' that guide learning, so in another sense they are not a blank slate.

factualpending

Speaker

Raphael Milliere

Evidence Quote

“you can think of this structure as embedding some priors, some biases, what's known in computer science as inductive biases that help learning in a specific domain.”

Source

230 | Raphaël Millière on How Artificial Intelligence Thinks— Sean Carroll's Mindscape
Created: 6/13/2026, 12:19:51 AM

Your notes

Loading notes...