Word embeddings—representing words as high-dimensional vectors learned through context prediction over trillions of tokens—capture semantic structure where similar words cluster together and directions in embedding space encode semantic relationships like the direction from king to queen being similar to the direction from man to woman, and going from present to past tense.

factualpending

Speaker

Justin Basilico

Evidence Quote

“king minus queen, that's the direction, is roughly the same as the direction you would go to get from from uh man to woman”

Source

Jeff Dean (Google): Exciting Trends in Machine Learning— Rice Ken Kennedy Institute
Created: 8/11/2026, 1:28:51 AM

Your notes

Loading notes...