cross-entropy
A standard, mathematically simple loss function used to train models to predict probabilities, e.g. next-token prediction.
As defined in Adam Marblestone – AI is missing something fundamental about the brain
Where it comes up · 1
Most important claims · 1 of 1Search all claims for “cross-entropy” →
- The AI field has neglected the role of very specific loss and cost functions; whereas machine learning prefers mathematically simple loss functions (next-token prediction, cross-entropy), evolution built enormous complexity into many different loss functions, turned on at different developmental stages, effectively a curriculum encoded as 'Python code' for what each brain region must learn.Adam Marblestone – AI is missing something fundamental about the brain