LLMs want to be helpful so strongly that if asked for output in a specific format without enough context, they will hallucinate rather than admit uncertainty; the solution is to provide an explicit 'escape hatch' telling the LLM to refuse and ask for clarification rather than guess.
causalpending
Evidence Quote
“the model really wants to actually help you so much that if you just tell it give me back output in this particular format even if it doesn't quite have the information it needs it'll actually just tell you what it thinks you want to hear and it's literally a hallucination.”
Created: 8/11/2026, 6:54:14 AM
Your notes
Loading notes...