Value functions—intermediate rewards that signal whether a system is on a promising path before reaching a final goal—are fundamental to efficient learning but are underutilized in current LLM training because long-horizon tasks make learning from final outcomes inefficient.