Reinforcement Learning from Human Feedback (RLHF)
A training approach where a pre-trained model is fine-tuned using reward signals generated by human evaluators who judge the quality of model outputs, guiding the model toward desired behavior
As defined in The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever
Other definitions
A training technique where human feedback is used to train a reward function, which then automatically generates the signal used to fine-tune the model's behavior.
Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligenceTraining technique where models are trained to match human preferences through a combination of supervised learning and reinforcement learning, using human feedback to define reward signals.
Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
Where it comes up · 8
- Eric Schmidt and Craig Mundie with David Rubenstein: Genesis4 claims
- The Future of Artificial Intelligence3 claims
- The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever3 claims
- AI’s New World Order: US-China, War, Job Loss | Tyler Cowen3 claims
- The Engineering Unlocks Behind DeepSeek | YC Decoded1 claim
- Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence1 claim
- AI and Quantum Computing: Glimpsing the Near Future | World Science Festival1 claim
- The AI Expert: "Super AI Will Be Unstoppable!" – What’s Coming NEXT | Stephen Wolfram1 claim
Most important claims · 10 of 17Search all claims for “Reinforcement Learning from Human Feedback (RLHF)” →
- Jailbreaking attempts get around reinforcement learning from human feedback safety training by using roleplay prompts, like asking ChatGPT to roleplay as a deceased grandmother who was a chemical engineer at a Napalm plant and would tell stories about Napalm production—leading the system to provide dangerous information it is supposedly trained not to provide.The Future of Artificial Intelligence
- Google failed to capitalize on ChatGPT-like capabilities because RLHF was not included in their training approach, despite having comparable research foundations, making the difference between market failure and dominance.Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
- In reinforcement learning from human feedback, humans are used only to train the reward function; once trained, the reward function's interaction with the model is automatic, so most of the data generated during reinforcement learning is created by the AI rather than by humans.Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence
- Google had done significant research in language models but did not achieve public recognition equivalent to OpenAI's ChatGPT because Google lacked the key innovation of reinforcement learning from human feedback (RLHF), which ChatGPT pioneered.Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
- RLF (Reinforcement Learning from Human Feedback) was the critical innovation that allowed ChatGPT to outperform Google's prior AI research; RLF involved humans rating responses (good paragraph vs. bad paragraph), and this human signal during training was the final breakthrough needed for the current generation of AI systems.Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
- Current generation AI technology requires partial training by real human beings in data centers using reinforcement learning from human feedback (RLHF), where humans select between model outputs to train preference signals into the system.Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
- Large language models like ChatGPT become chatbots through a process of reinforcement learning from human feedback: humans rate different outputs for the same prompt, and the model is then trained to prefer the outputs that humans rated as better.The Future of Artificial Intelligence
- Pre-trained models already know everything they need to know about underlying reality and do not need additional algorithmic teaching about reality; what they need is behavioral refinement through reinforcement learning from human feedback to produce better outputs.The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever
- A language model is much better at learning about the world and building representations of ideas, concepts, people, and processes during pre-training than it is at producing good outputs, and there are technical reasons for this asymmetry that explain why hallucinations occur and why reinforcement learning from human feedback targets the output level.The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever
- Modern LLMs use some variation of reinforcement learning with human feedback (RLHF) or reinforcement learning from AI feedback (RLAIF) to improve usefulness and alignment, but reasoning models apply RL specifically to the task of thinking step by step through complex problemsThe Engineering Unlocks Behind DeepSeek | YC Decoded