Recurrent Neural Networks (RNNs) preserve the order of tokens in a sequence by using an RNN cell initialized with a hidden state that updates at each time step by taking a token embedding as input and outputting a new embedding, with the hidden state encoding information about all previous tokens.

definitionpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“there is an RNN cell which is initialized with a random hidden State at the beginning at time step T it inputs a token embedding i t and uses it to update its internal hidden State embedding to HD and outputs an embedding OT in the next time step it inputs the next token updates its internal hidden State embedding again”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...