Claims

Large language models' capabilities in reasoning have advanced faster than Acemoglu expected; their capabilities are 'truly impressive.'

When Acemoglu uses ChatGPT, he is simultaneously surprised by how good it is in areas where he lacks expertise and disappointed by its unreliability when he knows the subject matter; it consistently pretends to know more than it does and gives incorrect answers through excessive extrapolation.

Current ChatGPT and similar general-purpose LLMs are unsuitable for professional domains like electrical work because they are: (1) not designed or optimized for the task, (2) unreliable, (3) not trained on domain-specific information about relevant equipment and standards, and (4) not trained on use cases of expert practitioners dealing with similar problems.