
10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT
What this covers
This is a single-narrator, chapter-by-chapter survey of roughly a decade of language-modelling research, organized around 50 concepts. It starts with basics such as tokenization and word embeddings, then moves through RNN (Recurrent Neural Network)) models and the gated variants, LSTM (Long Short-Term Memory)) and GRU. It covers the 2014 encoder-decoder design, whose weakness was squeezing a whole input into one vector. The 2015 attention mechanism, which lets the decoder weight individual encoder states through query-key scoring, is the first fix the video shows for that bottleneck.
The middle section explains why the Transformer displaced recurrence: self-attention gives every token a view of the whole sequence, and layers process tokens in parallel rather than step by step. The narrator also covers masked (causal) attention in the decoder and the cost of a fixed maximum sequence length. Workarounds for that limit include summarizing earlier output and fetching document chunks from vector databases like Pinecone. A separate line of research makes attention itself cheaper, with Longformer, Linformer and Performer trading some accuracy for lower memory use. Pretrained models follow, including BERT (Bidirectional Encoder Representations from Transformers)), XLNet, T5 and the GPT (Generative Pre-trained Transformer)) line.
The last stretch deals with alignment. GPT-3, at 175 billion parameters, showed few-shot and zero-shot behaviour by conditioning on prompts rather than updating weights. It also hallucinated, because its objective was plausible text, not correctness. The video walks through how OpenAI's InstructGPT responded: labelers wrote demonstrations and ranked outputs, a 6-billion-parameter reward model was trained on the rankings, and RLHF tuned the model against that reward. ChatGPT and GPT-4 come next. The video closes on the speaker's own open questions: what happens when an LLM is connected to the internet, whether these systems understand anything or only sound as if they do, and how people will come to trust a system that may outperform a single human.
The speaker traces the 10-year evolution of NLP research that culminated in 2023's AI boom, showing how foundational architectures (RNNs → Transformers), attention mechanisms, scale, and human alignment techniques (RLHF) progressively solved core problems in language modeling, ultimately enabling systems like ChatGPT to shift from task-specific fine-tuning to general instruction-following via prompting.
- RNNs and their gated variants (LSTMs, GRUs) initially addressed sequence processing but had long-term dependency and parallelization bottlenecks
- Attention mechanisms and Transformers solved both parallelization and long-term dependencies, enabling scale to billions of parameters
- Human alignment via RLHF and instruction tuning converted next-word prediction into controllable instruction-following, enabling zero/few-shot prompting as the primary interface
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
So far, LLMs had been trained using web data which gave them solid understanding of language semantics, syntax, and generation of fluent sentences, but GPT-3's objective was to predict likely sentences with no notion of correctness or factual bias, whereas the human goal in using an LLM is to generate coherent, informative, trustworthy, and reliable outputs.
“so far we had been using web data to train models which gave llms a solid understanding about the semantics and syntax of language and how to generate fluent sentences gpt3's objective was to predict likely sentences but there is no notion of correctness or factual bias into the model as humans our goal to query an llm is not to just generate fluent text but coherent informative trustworthy and reliable out outputs”
LLMs trained with human alignment via techniques like RLHF represent a paradigm shift, with one of the main reasons being their zero-shot and few-shot capabilities, where the baseline of a good LLM is no longer how easy it is to fine-tune on downstream tasks but how good it is at responding to prompts.
“llms trained with human alignment are truly a paradigm shift and one of the main reasons are because of their zero shot and few shot capabilities the Baseline of a good is no longer how easy it is to fine tune on some Downstream tasks but it is about how good it is to responding to prompts via prompting”
Reinforcement Learning from Human Feedback (RLHF) is a technique that fine-tunes an LLM's weights using a reward model to maximize the human-generated reward signal while retaining the semantic language understanding of the original model, and while RLHF has many use cases in reinforcement learning (especially in robotics), in the context of LLMs it fine-tunes the model to generate text that maximizes human reward.
“a technique known as reinforcement learning with human feedback was then used to further tune the model's weights using the rewards models rlhl had many use cases in the field of reinforcement learning especially in robotics but in a nutshell it fine-tunes gpt3 to generate text that maximizes the human generated reward signal output by the reward model while retaining the semantic language understanding of the original gpt3”
There is a whole line of research focused on reducing the computation cost of the attention module itself, often sacrificing some accuracy to speed up training and inference and significantly reduce memory requirements, with examples including Longformer, Linformer, and Performer.
“there's a whole line of research that's focused on reducing the computation cost of the attention module itself this often sacrifices a little bit of accuracy to speed up training and inference and significantly reduce memory requirements some examples of this models are long formers lint Farmers performers Etc”
GPT-3 showed that large models can be few-shot or zero-shot learners, where few-shot learning refers to the ability where the model is shown some demonstrations or examples of a task during inference as conditioning.
“gpd3 showed that large models can be few short or zero shot Learners few short learning refers to the ability where the model is shown some demonstrations or examples of a task during inference as conditioning”
GPT-3 demonstrated one-shot and zero-shot learning abilities, and unlike fine-tuning, few-shot learning does not update the network's weights but instead just conditions the model during inference with the prompt.
“some types of tasks dpt3 also demonstrated one shot and zero shot abilities note that unlike fine tuning fuse shot does not update the Network's weights and instead just conditions the model during inference with the prompt”
The seminal 2017 paper 'Attention is All You Need' introduced Transformers, which replaced RNNs entirely with attention mechanisms as encoder-decoder architectures, with the ripple effects of this innovation still felt today.
“a seminal paper called attention is all you need introduce the concept of a transformer the ripple effect of this paper is still felt today and lstms were about to be replaced the Transformers introducing attention is all you need are also encoder decoder architectures but instead of using any kind of rnns they used the attention mechanism”
One of GPT-3's major issues is hallucination, where the language model generates output containing incorrect, invented, or unrealistic information that was not present in the input, meaning these models can lie or fabricate information to please the user and are generally unsafe to use.
“one of the major ones is hallucination where a language model generates output that contains incorrect invented or unrealistic information that was not present in the input Hallucination is bad because these models can lie or fabricate information to please the user and are generally unsafe to use”
LSTMs and GRUs had two main problems: (1) long-term dependency issues where networks forget information from the distant past because they cannot encode everything into a hidden state vector, and (2) slow sequential training because each token must pass through the RNN cell one-by-one sequentially, with no parallelization possible since each token's output depends on everything that came before it.
“there were still two main issues with lstms and grus they still had long-term dependency issues where the network will tend to forget things that were said long time in the past because it simply can't encode everything into its hidden State vector and number two training was quite slow because we had to pass each token one by one sequentially through the RNN cell there is no way to parallelize this computation yet because each token's output embedding basically depend on everything that came before it”
Self-attention alleviates the long-term dependency issues of ordinary RNNs because models get a holistic view of the entire sequence and can attend to any token regardless of its position in the sequence.
“self-attention also alleviates the long-term dependency issues of ordinance because they get a holistic view of the entire sequence and can attend to any token regardless of their position in the sequence”
Gated RNNs (GRUs and LSTMs) add special gates to the basic RNN structure to selectively update and forget new information into and from the hidden state, showing massive improvement in the neural network's ability to learn with longer sequences compared to standard RNNs.
“grus and lstms are gated rnns that add special gates to the basic RNN structure to selectively update and forget new information into and from the hidden State this shows massive Improvement in the neural Network's ability to learn with longer sequences while gated variants are outperform normal rnns”
The main issue with the original encoder-decoder architecture is that the encoder must map the entire input sequence into a single vector embedding, which then becomes the sole source of information for the decoder to generate the entire target sequence, creating a severe information bottleneck.
“the main issue with the original sequence to sequence architecture is that the encoder has to map the entire sequence into a single Vector embedding which the decoder then has to produce the entire Target sequence from”
Decoder masked self-attention (causal attention) is critical because it ensures that during training the model can only attend to tokens that have already been generated and not to future tokens, preventing the model from cheating by predicting future tokens which would not be possible during inference.
“this is called decoder masked self-attention or causal attention and it is very important because it ensures that during training the model can only attend to tokens that have already been generated and not to Future tokens this is necessary to prevent the model from cheating and predicting future tokens which would not be possible during inference”
Each Transformer layer in both encoder and decoder can process all input tokens in parallel, speeding up training by a significant margin compared to LSTM training which had to step through each input sequentially.
“at training Time Each Transformer layer in the encoder and the decoder can process all the input tokens parallely speeding up training by a significant margin from lstm training which had to step through each input sequentially”
The attention mechanism (introduced in 2015) allows the decoder to selectively focus on specific tokens in the encoder sequence instead of relying on a single encoder output by having the decoder state act as a query vector, encoder states as key vectors, computing attention scores for each query-key pair, and forming context vectors as weighted averages of encoder states based on these scores.
“in 2015 we got the attention mechanism where the decoder can now selectively focus on specific tokens in the encoder sequence instead of relying on a single encoder output at each step the decoder can form new context vectors by combining relevant hidden States from the input sequence specifically the decoder State at each time step is treated as a query vector and all the encoder states are considered key vectors we learned special ways in the network to compute the attention score or the relevance score for each query decoder State Vector with respect to encoder key vectors depending on these attention scores we do a weighted average of the encoder vectors to form a special context Vector for each decoded query”
In InstructGPT, human labelers ranked multiple outputs generated by variants of GPT-3 according to their preference for a given input, and a much smaller reward model (6 billion parameters) was trained on these rankings to input a prompt and response and output a human reward score.
“next the human labelers were also asked to rank multiple outputs generated by variants of gpt3 according to their order of preference for a given input a much smaller reward model was trained on these rankings containing 6 billion parameters to input a prompt and a response and then output a human reward score”
A limitation of Transformers is that they must process entire sequences at once and therefore have to limit the maximum length of tokens they can process, with ongoing research into making Transformers have larger memory—such as by prompting with summaries of previous outputs or using vector databases like Pinecone to fetch relevant document chunks.
“a limitation of Transformers is that because they process entire sequences at once they have to limit the maximum length of tokens that they can process and they are still research going on about how to make Transformers have larger memory for example by prompting the model with a summary of its previous outputs or keeping a vector database like Pinecone to fetch smaller relevant parts from a large Corpus to then feed into the prompt”
This might be the first time in human history when we have encountered a program or entity that might be smarter than a single human, raising questions about how we will understand and learn to trust it absent propaganda, business interests, and clickbait competition.
“this might be the first time in human history when we have encountered a program or an entity which might be smarter than a single human how are we going to understand and learn to trust it without all the propaganda and the business and the race for an llm and all the clickbaits how do we learn to trust this and make it safe for us to use and shape our future”
ChatGPT was designed to have conversations with the user via a chat interface (contrasting with InstructGPT which was designed to provide guidance or follow instructions), was trained on a larger and more diverse dataset than InstructGPT (especially human-to-human conversations), and has been shown to be less hallucinatory and less biased than InstructGPT.
“a year after instruct GPT we got chat GPD PT was designed to provide guidance or follow instructions from the user while chat GPT is designed to have conversations with the user with a chat interface chat GPT was trained on a larger and more diverse data set than in stock GPT especially on human to human conversations and has been shown to be less hallucinatory and unbiased than instruct GPT”
The encoder-decoder architecture (introduced in 2014) solves sequence-to-sequence tasks by passing the entire input sequence token-by-token through an encoder RNN (LSTM or GRU), using the final hidden state as an embedding of the full input sequence, then passing this encoding as the initial hidden state to a decoder RNN that generates the target sequence.
“in 2014 we got this seminal paper to create an architecture for learning sequence to sequence tasks called the encoder decoder architecture we have an encoder which takes the input sequence and passes a token by token through an lstm or a GRU the final hidden state of the RNN after all tokens have passed represent the embedding of the input sequence this encoder's output Vector is then passed into the decoder as the initial hidden state of another RNN Network and the decoder then has to generate the target output sequence from this encoder output Vector”
GPT-3 advanced the GPT architecture of Transformer decoders with next-word prediction on a massive dataset with 175 billion parameters, making it by far the largest language model trained at that point.
“gpt3 Advanced the GPT architecture of Transformer decoders on next word prediction on a massive data set and 175 billion parameters by far the largest language model trained up to this point”
In 2022, OpenAI published InstructGPT, which collected prompts from OpenAI playground users (with consent) and had human labelers provide demonstrations of desired behavior as responses to the prompts, then fine-tuned GPT-3 on these annotated datasets.
“in 2022 openai published instruct GPD they collected a bunch of prompts from existing openai playground users after consent and got human labelers to provide demonstrations of the desired Behavior as response to the prompts then gpd3 was fine-tuned on these annotated data sets”
If LLMs are given the power to connect to the internet (which some open-source projects have already done), a key safety question emerges: how safe is it when an LLM is released into the wild interacting with people?
“if you give llms the power to connect to the internet with some open source projects already have how safe is it when an llm is out there into the wild interacting with people”
Despite their capabilities, LLMs remain black boxes where you can ask them something and they will answer, and from there you can infer their intelligence and understanding, but it's unclear whether they exhibit empathy and understanding through consciousness stored in their weights or are faking it to sound human and please the user.
“at the end of the day llms are still black boxes you can ask it something and it will answer you and from there you can infer about its intelligence and its understanding of the topic it might show empathy and Science Of Consciousness is it exhibiting those Tendencies because it has some notion of Consciousness stored within its billions of weights or is it faking it to you to sound like a human to please you”
T5 (Text-to-Text Transfer Transformer) introduced the idea of a unified model that can be trained on a variety of NLP tasks by framing them all as text-to-text problems, allowing the model to learn task-specific mappings through fine-tuning on the same model architecture.
“T5 or text to text transfer Transformer introduced the idea of a unified model that can be trained on a variety of NLP tasks by Framing them as a text to text problem and allowing the model to learn the task specific mapping through fine tuning you could prompt the same model to translate a sentence or summarize a passage or even sentiment analysis”
Language models are machine learning models designed to predict the likelihood of a sequence of tokens in a language.
“at the most basic level language models are machine learning models that are designed to predict the likelihood of a sequence of tokens in a language”
Word embeddings are dense low-dimensional vectors that represent each word in a model's vocabulary and capture semantic and syntactical structure such that similar words have their embeddings close to each other in vector space.
“we first convert each token into a vector word embedding word embeddings are dense low dimensional vectors that represent each word in the model's vocabulary word embeddings captures Mantic and syntactical structure of the words such that similar words have their embeddings close to each other”
Sequence-to-sequence tasks involve generating a sequence as output from another input sequence, with examples including machine translation (French to English) and summarization (long passage to short summary).
“one of the most common tasks studied in LLP is the sequence to sequence task this is a type of task where we need to generate a sequence as output from another input sequence it could be machine translation where the input is a French sentence and output is an English sentence or summarization where the input is a long passage and the output is a short summary”
GPT-3 can perform many tasks: write long passages of coherent text, correct grammar, and translate sentences.
“so gpd3 can do a lot of smart things you can write long passages of coherent text it can correct grammar it can translate sentences”
GPT models follow a Transformer decoder-like architecture and are trained on large corpuses of web data using next-word prediction as the training objective.
“gpts follow a Transformer decoder-like architecture that are also trained on large corpuses of web data with the task of next word prediction”
Next-word prediction is a training task in which the model is trained to predict the most likely next word that comes after a sequence of context words.
“next word prediction is a task in which the model is trained to predict the most likely next word that comes after a sequence of context words”
BERT uses a Transformer encoder architecture and is a pre-trained language model designed to produce contextualized sentence embeddings.
“bird uses a Transformer encoder architecture and is a pre-trained language model to produce contextualized sentence embeddings”
Recurrent Neural Networks (RNNs) preserve the order of tokens in a sequence by using an RNN cell initialized with a hidden state that updates at each time step by taking a token embedding as input and outputting a new embedding, with the hidden state encoding information about all previous tokens.
“neural networks that are able to preserve the order of tokens in a sequence when creating embeddings in a nutshell the rnns work as follows there is an RNN cell which is initialized with a random hidden State at the beginning at time step T it inputs a token embedding i t and uses it to update its internal hidden State embedding to HD and outputs an embedding OT in the next time step it inputs the next token updates its internal hidden State embedding again and produces another output token for the next time step”
Tokenization splits raw text into multiple tokens which are then inputted for processing into a machine learning model, with most modern models using subword tokenizers that split rare words into smaller sub-word units while frequently occurring words are inserted as whole units.
“in order for us to insert text into a machine learning model we first need to do tokenization where we split the raw text into multiple tokens which are then inputted for processing into an ml model most models today use a subword tokenizer that splits rare words into smaller sub word units while frequently occurring words are inserted as whole for processing”
Bidirectional RNNs make two passes through data—one left-to-right and one right-to-left—producing two separate embeddings that are then added or concatenated to form a bidirectional embedding that captures context from both directions.
“bi-directional rnns make two passes through the data once from left to right and then from right to left to produce two separate embeddings that are then added or concated together to form a bi-directional embedding”
During inference in sequence-to-sequence models, each word that the model outputs is fed back as the input word in the next time step to generate an entire sentence until an end token is predicted.
“during inference we feed e each word that the model outputs back as the input word in the next time step and generate an entire sentence until the end token has been predicted”
Beam search is a decoding mechanism that looks back multiple time steps into the future rather than just one time step, keeping track of a fixed number of top-scoring tokens at each step and expanding them in a branch-and-bound fashion until the end of the sequence, producing better sentences than greedy single-token prediction.
“readily predicting the most likely token at each time step may not always result in the best sentence so often we employ a beam surge mechanism where the model looks back multiple time steps in the future instead of just one to find the most likely sequence of words by keeping track of a fixed number of top scoring tokens at each step and expanding them in a branch and bound like fashion until the end of the sequence is reached”
Self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence, allowing the model to weigh the importance of different tokens based on their contextual relevance to each other.
“all the input token embeddings are updated using a concept called self-attention self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence allowing the model to weigh the importance of different tokens based on their contextual relevance to each other”
Training the network to learn attention weights allows the network to selectively focus on specific parts of the encoder sequence when generating specific parts of the target sentence, as evidenced by attention heatmaps showing specific English words focusing on corresponding French words in machine translation.
“allowing the network to train these special weights allows the network to selectively focus on specific parts of the encoder sequence when generating specific parts of the target sentence as can be seen by these heat Maps where specific English words can be seen as providing focus on corresponding French words while ignoring other parts of the French sentence”
Transformers introduced multi-headed attention where each query vector learns multiple attention maps for each key vector and produces multiple context vectors instead of just one, allowing the model to simultaneously attend to different types of relationships.
“Transformers introducing attention is all you need are also encoder decoder architectures but instead of using any kind of rnns they used the attention mechanism specifically they introduced the concept of multi-headed attention where each query Vector learns multiple attention maps for each of the key vectors and produces multiple context vectors instead of just one”
GPT was considered the first large language model with over 117 million parameters trained on text data acquired from the web.
“GPT is considered the first llm it had over 117 million parameters and trained on text Data acquired from the web”
To retain sequential information in Transformers, special positional encodings are trained that represent specific indices in the sequence and are added to the corresponding word embeddings to form the final input token embedding.
“to retain the sequential information Transformer strain special high frequency dense embeddings which represent a specific index in the sequence these positional encodings get added to the corresponding word embedding to form the final input token embedding”
Large Language Models (LLMs) are deep learning models designed to process natural language with typically hundreds of millions or more parameters trained on huge corpuses of text data, enabling them to capture complex relationships between words and generate coherent text.
“llm stands for large language model and a deep learning models to process natural language these models typically have a large number of parameters usually above a few hundred millions and are trained on huge corpuses of text Data allowing them to capture complex relationships between the words as well as generate coherent text”
Knowledge distillation is a process where a large complex model (like BERT or GPT) is used to train a smaller simpler model to mimic its behavior, resulting in smaller models that run much faster than the original and can often be used on consumer machines.
“knowledge distillation is a process where a large complex model like bird or GPT is used to train a smaller simpler model to mimic Its Behavior that run much faster than the original models and can often be used on consumer machines the smaller model trains from the same input data and learns to predict output probabilities and intermediate States similar to the larger Model”
Instead of using absolute positional embeddings as in the original Transformer, relative positional embeddings enable the model to learn positional information of tokens relative to each other and allow attention to tokens in only a fixed-length context window.
“instead of the absolute positional embeddings the original Transformer applied to each token Transformer Excel introduced the idea of relative positional embeddings which enables the model to learn positional information of tokens relative to each other and pay attention to tokens appearing in only a fixed length context window instead of the entire sequence”
XLNet is a pre-training language model that combines the autoregressive training of GPT and the bi-directional nature of BERT to introduce permutation language modeling, which predicts the likelihood of a token given the entire input sequence in any random order of appearance.
“excellent is a pre-training language model that combines the autoregressive training of GPT and bi-directional nature of birth to introduce the concept of permutation language modeling excellent predicts the likelihood of a token given the entire input sequence per factor in any random order of appearance”
XLNet beat BERT on many fine-tuning downstream tasks by a large margin.
“excellent beat bird in many fine-tuning Downstream tasks and by a large margin too”
Low-Rank Adaptation (LoRA) is a fine-tuning technique that freezes all pre-trained model weights and adds trainable rank decomposition matrices into each layer of the Transformer architecture, reducing the number of trainable parameters during fine-tuning by about 10,000 times as well as GPU requirements.
“low rank adaptation or Laura is a fine tuning technique that freezes all the pre-trained model weights and adds trainable rank decomposition matrices into each layer of the Transformer architecture this greatly reduces the number of trainable parameters during fine tuning by about 10 000 times as well as the GPU requirements”
You can make GPT-3.5 or GPT-4 assume a role, ask it to do zero-shot tasks, make it output creative poems, and provide examples in the prompt to do few-shot inferences without needing to update its weights.
“you can make gbd 3.5 or gbd4 assume a role ask it to do zero short tasks make it output creative poems and provide some examples in the prompt to do few short inferences without needing to update its weight”
The decoder in Transformers uses two types of attention: (1) cross-attention where the decoder embedding acts as query and attends to encoder state embeddings, and (2) masked self-attention (causal attention) where the decoder attends to target sequence tokens but only those that have appeared in the past, with future tokens masked away.
“the decoder uses two types of attention much like the decoder architectures of lstm based sequence sequence models the decoder embedding is treated as the query and provides attention to the encoder's state embeddings the decoder also provides attention to the embeddings of the target sequence however only to tokens that have appeared in the past the tokens in the future are masked away this is called decoder masked self-attention or causal attention”
One of BERT's pre-training tasks was masked language modeling, where random tokens are masked from the input sequence and the network is tasked to predict these tokens using bi-directional self-attention on all other tokens in the sequence.
“one of the pre-training tasks bird was trained on was called mask language modeling random tokens are masked from the input sequence and the network is tasked to predict these tokens by providing bi-directional self-attention on all the other tokens in the sequence”
GPT-4 is an order of magnitude larger than GPT-3 and is generally more truthful, safe, creative, collaborative, and reliable than any other language model.
“gpd4 is an order of magnitude larger than gpt3 and is generally more truthful safe creative collaborative and reliable than any other language model out there”