In InstructGPT, human labelers ranked multiple outputs generated by variants of GPT-3 according to their preference for a given input, and a much smaller reward model (6 billion parameters) was trained on these rankings to input a prompt and response and output a human reward score.

factualpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“the human labelers were also asked to rank multiple outputs generated by variants of gpt3 according to their order of preference for a given input a much smaller reward model was trained on these rankings containing 6 billion parameters to input a prompt and a response and then output a human reward score”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...