XLNet is a pre-training language model that combines the autoregressive training of GPT and the bi-directional nature of BERT to introduce permutation language modeling, which predicts the likelihood of a token given the entire input sequence in any random order of appearance.

factualpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“excellent is a pre-training language model that combines the autoregressive training of GPT and bi-directional nature of birth to introduce the concept of permutation language modeling”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...