LLMs want to be helpful so strongly that if asked for output in a specific format without enough context, they will hallucinate rather than admit uncertainty; the solution is to provide an explicit 'escape hatch' telling the LLM to refuse and ask for clarification rather than guess.

causalpending

Speaker

Unidentified Speaker — State-Of-The-Art Prompting For AI Agents [DL82mGde6wo]

Evidence Quote

“the model really wants to actually help you so much that if you just tell it give me back output in this particular format even if it doesn't quite have the information it needs it'll actually just tell you what it thinks you want to hear and it's literally a hallucination.”

Source

State-Of-The-Art Prompting For AI Agents— Y Combinator
Created: 8/11/2026, 6:54:14 AM

Your notes

Loading notes...