factual

Evaluating Understanding Is an Unsolved Problem

AI researchers do not yet know how to precisely define or evaluate understanding, which has become a new challenge because the field now moves fast enough that evaluation must be redesigned; this difficulty parallels how we lack good single measures for evaluating humans, so collectively assessing capabilities may be necessary.

factualpending

Speaker

Yejin Choi

Evidence Quote

“we don't even know how to define understanding precisely and it's been rather a moving target”

Source

248 | Yejin Choi on AI and Common Sense— Sean Carroll's Mindscape
Created: 6/13/2026, 12:23:11 AM

Your notes

Loading notes...