post-training
Training applied after pre-training, using task-specific or expert-curated datasets to refine a model's behavior for particular domains.
As defined in Brendan Foody
Other definitions
A phase of AI model development that occurs after initial pre-training, including supervised fine-tuning, reinforcement learning from human feedback, and other techniques to align models with human preferences and improve performance on downstream tasks.
Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad GerstnerA phase of model development after initial pre-training where models are fine-tuned on specific tasks, domains, or behaviors using additional training techniques like reinforcement learning or supervised fine-tuning
How Claude's AI Financial Analyst is Changing Investing (Insider Unveils)
Where it comes up · 20
- John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI5 claims
- Christina Cacioppo - Building an Enduring Software Company in the Age of AI - [Invest Like the Best, EP.496]4 claims
- Eric Vishria - A Decade of Lessons Investing in Software & Hardware - [Invest Like the Best, EP.486]2 claims
- Martin Casado - Entering Uncharted AI Territory - [Invest Like the Best, EP.381]2 claims
- Sarah Guo - Funding the Frontier - [Invest Like the Best, EP.489]2 claims
- Lex Fridman Podcast #491 - OpenClaw2 claims
- Perplexity CEO: Disrupting Google Search with AI1 claim
- Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner1 claim
and 12 more
Most important claims · 10 of 31Search all claims for “post-training” →
- Pre-training scaling laws are now being supplemented by post-training scaling, multimodality scaling, synthetic data generation, and inference-time scaling, with inference-time scaling now scaling up to achieve intelligence through reasoning, simulation, and reflection rather than just model size.Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
- The labor that goes into training and post-training generative AI models is deliberately offshored to developing countries where workers endure trauma-inducing exposure to violent and hateful content, and companies exploit vulnerable populations including refugees, people in countries with hyperinflation, and prison laborers who lack labor protections.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- Models will struggle in law for a long time because much of the right way to approach legal problems is not written down or codified—it exists in the heads of experts—and without those tokens in pre-training data or expert-created post-training data, the model will inevitably struggle.Brendan Foody
- Post-training is somewhat of a moat because it is a complex operation requiring much skilled, tacit, and organizational knowledge and accumulated R&D that's non-trivial to spin up; but distillation—cloning another model's outputs or using it as a judge—undercuts this moat, and smaller players likely do this to catch up despite it violating terms of service.John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI
- The rapid advancement in AI happens not in architecture but in the training stages—pre-training, mid-training, and post-training—where capability unlocks like supervised fine-tuning and RLHF (which turned GPT-3 into ChatGPT) come from the algorithmic side rather than architectural change.Ai Sota #2026
- Pre-training trains a model to imitate all internet content by maximizing log probability of the next token, which forces it to assign probabilities to everything and ends up making the model very calibrated, capable of both generating and assigning probabilities to all kinds of web content and personas.John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI
- Post-training narrows the model from a base model that can take on any persona to a specific helpful chat-assistant persona, optimizing on a different objective—producing outputs humans will like and find useful—rather than imitating raw web content.John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI
- Over the last three years open-source models have become increasingly competitive from China, the US, and Europe, with recent examples including Thinky, Poolside, NVIDIA models, and Mistral, and Guo argues the capability is already present and widely used.Sarah Guo - Funding the Frontier - [Invest Like the Best, EP.489]
- The roughly 100-point Elo score improvement of current GPT-4 over the originally released version came mostly from post-training, driven by multiple stacking axes—data quality, data quantity, more iterations of deploying and collecting new data, and changing annotation types—that together yield a substantial effective compute increase.John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI
- The latest generation of models have extensive post-training that makes them more resilient to prompt injection, so simple 'ignore all previous instructions' attacks no longer work and weak or cheap local models should be avoided because they are very gullible and easy to inject.Lex Fridman Podcast #491 - OpenClaw