The current architecture of large language models may create hard limits on reliability that could be unacceptable in high-stakes domains like medicine; for example, a 1-in-1,000 error rate in nursing recommendations would be unacceptably high, as it would constitute an unacceptable casualty rate in medical applications.