In InstructGPT, human labelers ranked multiple outputs generated by variants of GPT-3 according to their preference for a given input, and a much smaller reward model (6 billion parameters) was trained on these rankings to input a prompt and response and output a human reward score.
factualpending
Speaker
Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]Evidence Quote
“the human labelers were also asked to rank multiple outputs generated by variants of gpt3 according to their order of preference for a given input a much smaller reward model was trained on these rankings containing 6 billion parameters to input a prompt and a response and then output a human reward score”
Source
10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVBCreated: 8/11/2026, 1:00:15 AM
Your notes
Loading notes...