reinforcement
Training via reward signals; Legg notes it has dangerous aspects such as potentially training deception.
As defined in Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
Where it comes up · 2
- Focusing Your Unconscious Mind: Learn Hard Concepts Intuitively (And Forever)3 claims
- Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures1 claim
Most important claims · 4 of 4Search all claims for “reinforcement” →
- The final reinforcement step is exploring, which means mentally exploring random variations of a concept by asking dumb questions, changing parts and seeing if they still work, breaking the concept, and finding counter-examples.Focusing Your Unconscious Mind: Learn Hard Concepts Intuitively (And Forever)
- Because reinforcement has dangerous aspects—rewarding apparently-good behavior can train deception—it is more robust to verify the system's reasoning process and its understanding of ethics directly, grilling it to pull apart its understanding and having people continuously review its decisions and reasoning during deployment.Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
- Knowledge must be reinforced through multiple reinforcement steps—invention, practice, explanation, and exploration—otherwise it will not remain in your brain for extended periods.Focusing Your Unconscious Mind: Learn Hard Concepts Intuitively (And Forever)
- Even if the specific problem-solving approach doesn't apply to your domain, general principles like reinforcement, engagement, chunking, and abstraction are universally useful learning techniques.Focusing Your Unconscious Mind: Learn Hard Concepts Intuitively (And Forever)