Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargontechnology· 3 claims· 2 episodes

Pre-trained

A model that has been trained on data before being deployed or released; the P in GPT; current models are frozen after pre-training and fine-tuning, though future models may learn continuously

As defined in Generative AI in a Nutshell - how to survive and thrive in the age of AI


Where it comes up · 2
  • What Is ChatGPT Doing? Episode 1: Introduction2 claims
  • Generative AI in a Nutshell - how to survive and thrive in the age of AI1 claim

Most important claims · 3 of 3Search all claims for “Pre-trained” →
  • Pre-trained means the model learns from examples by being shown large amounts of text content from diverse sources including Wikipedia articles, book databases, Reddit posts, cooking recipes, and relationship discussions, processing sentence by sentence to predict what comes nextWhat Is ChatGPT Doing? Episode 1: Introduction
  • ChatGPT stands for Generative Pre-trained Transformer and is the neural network architecture on which the whole system is builtWhat Is ChatGPT Doing? Episode 1: Introduction
  • After training is complete, LLM models are mostly frozen, with only limited fine-tuning occurring thereafter; the 'P' in GPT stands for 'pre-trained', indicating that the model's parameters are set during training rather than continuously updated in deployment.Generative AI in a Nutshell - how to survive and thrive in the age of AI