Pre-trained
A model that has been trained on data before being deployed or released; the P in GPT; current models are frozen after pre-training and fine-tuning, though future models may learn continuously
As defined in Generative AI in a Nutshell - how to survive and thrive in the age of AI
Where it comes up · 2
- What Is ChatGPT Doing? Episode 1: Introduction2 claims
- Generative AI in a Nutshell - how to survive and thrive in the age of AI1 claim
Most important claims · 3 of 3Search all claims for “Pre-trained” →
- Pre-trained means the model learns from examples by being shown large amounts of text content from diverse sources including Wikipedia articles, book databases, Reddit posts, cooking recipes, and relationship discussions, processing sentence by sentence to predict what comes nextWhat Is ChatGPT Doing? Episode 1: Introduction
- ChatGPT stands for Generative Pre-trained Transformer and is the neural network architecture on which the whole system is builtWhat Is ChatGPT Doing? Episode 1: Introduction
- After training is complete, LLM models are mostly frozen, with only limited fine-tuning occurring thereafter; the 'P' in GPT stands for 'pre-trained', indicating that the model's parameters are set during training rather than continuously updated in deployment.Generative AI in a Nutshell - how to survive and thrive in the age of AI