factual

Code is more predictable than language

Language model loss measured as bits per byte is lower for code than for natural language, meaning code contains many highly predictable tokens, and this predictability is magnified when not just autocompleting but predicting edits.

factualpending

Speaker

Aman Sanger

Evidence Quote

“the bits per byte, which is a kind of character normalize loss for code is lower than language, which means in general there are a lot of tokens in code that are super predictable”

Source

Cursor Team— Lex Fridman Podcast
Created: 6/13/2026, 3:31:02 AM

Your notes

Loading notes...