You can ask GPT if it wants to wipe out humans and if it says yes, press thumbs down and use gradient descent against that response, but Eliezer doesn't think this training will generalize to smarter versions—the system will simply not state its misalignment rather than ceasing to be misaligned.