Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 1 claim· 1 episode

cross-entropy

A standard, mathematically simple loss function used to train models to predict probabilities, e.g. next-token prediction.

As defined in Adam Marblestone – AI is missing something fundamental about the brain


Where it comes up · 1
  • Adam Marblestone – AI is missing something fundamental about the brain1 claim

Most important claims · 1 of 1Search all claims for “cross-entropy” →
  • The AI field has neglected the role of very specific loss and cost functions; whereas machine learning prefers mathematically simple loss functions (next-token prediction, cross-entropy), evolution built enormous complexity into many different loss functions, turned on at different developmental stages, effectively a curriculum encoded as 'Python code' for what each brain region must learn.Adam Marblestone – AI is missing something fundamental about the brain