factual

Prompt Space Is Too Large to Fully Fine-Tune

The space of all possible prompts is so gigantic that fine-tuning can only cover a tiny subset; there is a long tail of prompts the system was never trained on, so an LLM can always be broken into producing nonsense, making it effectively a giant lookup table rather than a reasoning system.

factualpending

Speaker

Yann LeCun

Evidence Quote

“the system ends up being a giant lookup table essentially, which is not really what you want, you want systems that can reason”

Source

Research Pioneer— Lex Fridman Podcast
Created: 6/13/2026, 3:30:53 AM

Your notes

Loading notes...