In 1985, Hinton created a small language model (112 training cases initially, then ~1000) trained to predict the next word to learn representations of word meanings, which demonstrated that feature vectors for word meanings and interactions between features could explain how meaning relates to other words, reconciling the psychological feature-vector theory and structuralist relational theories of meaning.
factualpending
Speaker
Geoffrey HintonEvidence Quote
“back in 1985 I made a little language model you just make the first L lowercase um it had like 112 training cases and it had a neural net with several thousand weights um it Lear it was the first Model that was trained to get representations for the meanings of words by trying to predict the next word in a sequence”
Source
Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024— Vector InstituteCreated: 8/11/2026, 6:32:35 AM
Your notes
Loading notes...