
AI, Productivity and the Surprising Risk No One’s Talking About | Merryn Talks Money
What this covers
This is a conversation between Merryn Somerset Webb and Tom Slater, who manages Scottish Mortgage Investment Trust at Baillie Gifford, built around his paper on AI and the mind. Slater starts from cultural evolution. Learning to read repurposed the part of the brain used for recognizing faces, and he argues that culture can change physiology faster than genes can. He frames AI as boosting variation, transmission and selection of ideas: it generates far more of them, funnels distribution through two or three dominant systems, and lets algorithms decide which ideas win. From there the talk moves to the central paradox of AI, in which output improves today while the human capacity behind it weakens.
The evidence is mostly studies and professional examples. An MIT experiment found that over 80% of students who wrote essays with AI help could not quote their own work, which Slater takes as a failure of durable learning. A group that used search engines behaved very differently. In endoscopy, detection rates fell 21% once experienced practitioners lost their AI, and they went along with confident wrong answers. Merryn adds lawyers citing invented cases and the pilot comparison with autopilot. Slater counters that AI can improve experts when used deliberately, citing human players who developed new strategies in Go after the AI's win.
The later sections turn practical and more contentious. They discuss the hollowing out of entry-level white-collar work, and ask how people can judge AI output if the base skills are never built. Slater advises young people to take the hard route rather than the shortcut. On companies, he calls cutting investment in human skill a false economy, though survival pressure makes that hard to resist. He expects inequality to widen between firms and between countries, with the US and China holding the leading systems and the UK importing them. Merryn presses the point that a university degree bought with heavy debt and completed through AI could be worse than pointless. Slater is cautiously optimistic that digital natives will handle AI better than earlier adopters.
AI reshapes human cognition and capability through passive use, creating a central paradox where immediate task performance improves while underlying human skills atrophy, leading to widening inequality unless individuals and organizations deliberately engage with these tools while maintaining foundational expertise.
- Reading rewired human brains biologically across just 200 years, demonstrating culture can hijack physiology faster than evolution; AI adoption is accelerating this process by orders of magnitude
- MIT study showed 80% of students using AI assistance couldn't recall their own essays and conflated tool mastery with subject knowledge, breaking the learning loop that builds expertise
- Endoscopy study revealed even experienced practitioners overrelied on confident but incorrect AI outputs, and detection rates fell 21% after AI access was removed, showing capability erosion
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Culture hijacks physiology and changes the way you think, and learning to read and write caused biological changes in the brain by repurposing the area used for recognizing faces, creating a trade-off where humans became less good at face recognition but gained an incredible skill.
“culture hijacks your physiology and changes changes the way you think. And, um, in particular, this learning the skill of, of, um, reading and writing. You know, it it caused these changes in the brain. Um, you know, we we repurposed this area of the brain that was, was used for recognizing faces. And there's a bit of a trade off there.”
You have to learn how to think fundamentally to be employable, and that's what you need to go to university to learn to do, not just to acquire facts but to develop the cognitive capacity to think independently.
“And but I do I do think that you have to learn how to think fundamentally to be employable. You have to be able to think for yourself. Yeah. And that's what you need to go to university to learn to do.”
There isn't a great deal of value in something that benefits the next handful of years at the expense of all future years because saving money in the short run creates a real long-term cost to your ability to operate your business, making it a false economy.
“There isn't a great deal of value in something that benefits the next handful of years at the expense of all future years... if it's going to have a real long term cost to your ability to, um, operate your business, then that that's a false economy.”
At the company level, the real challenge is that companies need to keep paying to ensure their employees have a base of knowledge and skill, but this is very hard because there's a constant battle for survival in the corporate world where you have to be cost-effective.
“I think we we've we've slightly come up with the answer to that. They need to keep paying up to make sure that their employees have a base of knowledge and skill beyond that. But I do think it's it's really hard because, you know, you have this, um, this constant battle for survival in the corporate world. You have to be cost effective.”
An MIT study found that when students wrote essays with AI assistance, over 80% of that group couldn't provide a single correct quote from the essay they had just written a few minutes earlier, showing that passive AI use bypasses the durable learning that comes from effortful writing.
“So what you saw was that, um, if you looked at, um, what happened to the people, um, if, if, if they were given I, they would essentially outsource the writing process. And the result was that about a over 80% of of the that group that used AI to write the essay couldn't provide a single correct quote from the essay they had just written a few minutes earlier.”
When experienced endoscopy practitioners used AI and then had their AI systems removed, their detection rate fell 21% for the same group, and when experimenters tweaked the AI to give confident but incorrect interpretations, human performance collapsed as experienced practitioners accepted the wrong answers.
“If you had these experienced practitioners, um, using AI systems, um, and then after you took their AI systems away, um, there was one key indicator, um, for example, where the detection rate fell 21% for the same group before and after they, they'd had access to these AI tools. Um, but then there was this, and I think this is probably the more worrying result was that, um, people tend to trust the output from the AI. Um, and so when, when the experimenters went in and um, um, tweaked the AI to give confident but incorrect interpretations, what you saw was that human performance collapsed.”
In the game of Go, after AI famously beat a human player, human players learned from the AI and developed novel strategies and novel ways of playing the game, showing that in expert domains, AI can make you better when used deliberately to be complementary to human behavior.
“So, you know, one one example of that was in the, in the game go, where AI famously beat a human player. Um, but what you've seen and subsequent to that, um, that sort of landmark event was that human players have learned from the AI, they it's developed novel strategies, novel ways of going about the game. And you've seen an improvement in human performance.”
The big conversation about AI is around employment and the disappearance of junior jobs, but this isn't theoretical—we are already seeing the hollowing out of white-collar professions happening in real time.
“The big conversation, of course, at the moment is around employment and about the disappearance of junior jobs, which we talked about just now. But that's not theoretical that that's happening.”
There is a bias toward older pilots who have strong base knowledge about how to actually fly a plane because they had to do it for a good while without autopilot before now, meaning years of manual practice built expertise that newer pilots might lack.
“this is why everyone has a bias towards older pilots, right? I mean, I don't know about you, but when I got on an aeroplane airplane and I see that the pilots are kind of maybe over 45 or better still over 50, I'm thrilled, absolutely thrilled, because I know that it means that they've got a strong base of knowledge about how to actually fly a plane, because they probably had to do it for a good while, um, without autopilot before, uh, before now.”
The group that merely used search engines (without AI) performed completely differently from the AI group, suggesting that AI's passive, answer-generation approach is fundamentally different from active research and learning.
“that was completely different in the group that you search the, the group that that wrote themselves.”
In the legal industry, lawyers have relied on AI that produced false quotes, false cases, and false precedent, showing that overreliance on confident but incorrect AI outputs is not limited to hypothetical studies but is happening in real professional settings now.
“Yeah. And we've seen we've seen this happened in these studies in the in the medical industry are really intrigued. But we've seen this happen in real life in the legal industry, haven't we. We've seen lawyers actually on several occasions. In fact, we've had examples of this in Scotland where I live, seen lawyers actually relying on I am producing false stuff, false quotes, false cases, false precedent.”
The challenge with the AI transition is that it's happening so fast that in previous transitions, you had time to work through these issues, but we don't have a lot of time because this is coming at us very quickly.
“The and the challenge with it comes that it's happening so fast. You know, in previous transitions, you've had time to work through these issues. And, and and the challenge here is that we don't have a lot of time because this is coming at us so quickly.”
Meta was paying individuals hundreds of millions of dollars for their superintelligence AI systems unit because they wanted a small group of engineers who could keep the whole AI system in their head, and even though hundreds of millions is expensive, it's dwarfed by the amount they're spending on hardware.
“I do take the example of, of Meta the, the owners of Facebook, you know, the it was, it was last summer that they were going, you know, they, they went through this hiring spree, um, uh, for, for their, their superintelligence, their AI systems unit. And the rumors were that they were paying, you know, individuals hundreds of millions of dollars to come and do this. And the rationale was that, you know, you're spending billions of dollars on these systems. Um, you what you want is a small group of engineers who can really keep that whole system in their head. And even if you're paying them hundreds of millions of dollars, it that sort of is dwarfed by the amount that you're spending on the hardware.”
The central paradox of AI is that it reliably improves immediate task performance while degrading the underlying human capabilities that produce that performance, meaning you get better results today but become less capable tomorrow.
“So a quote from from your paper, the central paradox is this AI reliably improves immediate task performance while degrading the underlying human capabilities that produce that performance. You get better results today but become less capable tomorrow.”
AI isn't coming for your job—it's coming for your mind, because the technology reshapes the way we think and changes our brains at a speed incomprehensible compared to evolutionary timescales, operating at cultural rather than genetic speeds.
“AI Isn't Coming for Your Job. It's Coming for Your Mind... it's about how AI is reshaping the way we think. At a fairly extraordinary speed.”
If the job market starts at a higher level (eliminating entry-level positions), the base knowledge required for those higher-level roles is never embedded and created, meaning the next generation can't judge whether AI output is correct because they lack foundational skills to evaluate it.
“if the job market starts at the next level up, that base knowledge is never embedded and created. You have to judge whether what your agent has told you is correct or not, but you no longer have the foundational skills to be able to do that.”
AI-driven inequality spreads across companies: companies that adopt AI and win competitively pull ahead of those that don't, creating inequality between companies, then between countries where US and China have dominant AI systems while UK depends on importing them, and cheap power becomes another factor that widens inequality.
“Um, and you get that within a single company, but then you start to get that across companies, you know, which are the ones that have been able to adopt these tools and, and, um, embrace them and, and win competitively. And, and so you get the inequality between these two companies. And then, you know, it's in between countries as well because you know, you know who's who's AI we're using, you know, in, you know, in the UK where we mostly using AI systems from the US and, and, and from uh, from China and, and and cheap power then becomes another really important factor in that. And we have very expensive power. So you can you can see these, these full lines of inequality and widening rapidly.”
Cultural changes operate with the same three mechanisms as genetic changes: variation (the number of ideas that can be tested), transmission (how ideas spread), and selection (which ideas get amplified), and AI supercharges all three by exponentially increasing idea generation, centralizing information distribution through 2-3 dominant AI systems, and allowing algorithms to control which ideas succeed.
“And the three factors of variation, transmission and selection. Um, so, you know, if you think about, um, if you go back to the weirdest people in the world, the example there is the Catholic Church and how, you know, that controlled variation and idea, is it control transmission, the way ideas spread and it controlled selection. Um, now, if you if you if you apply that to what's going on with in the world today, um, you know, AI, um, we can supercharge variation because, you know, the number of ideas that you can test, that you can generate, you know, grows exponentially and ideas that humans just wouldn't have got to. Um, it completely changes transmission of ideas. Um, so, you know, if if a million children, um, ask their parents, why is the sky blue? They'll get close to a million different answers. Um, but because we're all start, it started using 2 or 3 of these really dominant systems. Now everybody's getting some variation of the same answer.”
Young people should not take shortcuts to productivity that AI offers; instead, they should invest in the hard yards—doing things they don't want to do, effortful struggle—because that builds capabilities most important over the long run.
“don't take the easy path. Don't, um, you know, take the shortcuts to productivity that these tools give you. It's it's putting in the hard yards. It's doing the things that you don't want to do that effortful struggle, um, and generate, you know, um, build investing in yourself and your own capabilities. Um, that will be most important over the long run.”
It is possible to use AI as an always-on, infinitely patient tutor that answers every question, though sometimes incorrectly, creating a much better learning experience than traditional university.
“effectively you have a an always on, infinitely patient tutor who can answer you every question. Um, you know, it sometimes incorrectly.”
There's a tendency to give AI systems false and inappropriate amounts of respect, accepting their outputs as true when they confidently state things about the world, meaning your relationship with AI tools changes as they become more effective and their confidence becomes a source of unwarranted deference.
“there just isn't this stable balance between us and the machines is, you know, as you as you use these tools as, as they become more and more effective, um, your relationship with them changes. And, you know, there is we have this tendency to, to in it when they confidently say things about the world to accept it is true. We give them a false, um, and, um, an inappropriate amount of respect.”
If you engage positively with AI systems by using them as a tool to research and understand questions rather than outsourcing the entire task, you get the same cognitive results as if you had done the work yourself, meaning effortful interaction with AI output is the crucial factor in developing cognitive skills.
“But if you engage, um, positively with these systems so you don't you have the essay to write. You don't just say to the AI write the essay, but you go and research the question using AI as a tool. You know, I want to understand why this happened... you actually interact with it and think about the output and engage with it. Then you get the same results, you know, cognitively as if you if you just handwritten the essay.”
There is real value in having humans learn to do white-collar tasks without access to AI systems so that you retain the ability to oversee and override these systems when they're not producing the right answers, requiring investment in human capital first.
“I think there's real value in having humans learning to do these, these tasks, you know, which have white collar profession, you know, and having them do it without access to these systems. Um, and so that you, you, you then retain that ability to, you know, oversee, override these systems when, when they're not producing the right answers. Um, but you have to invest in the human capital first.”
The university cost problem combines with AI risk: students paying extraordinary student loan costs while using AI to externalize their learning means they've invested hundreds of thousands in an education that doesn't actually teach them anything, making the entire university experience not just pointless but worse than pointless.
“Kids going to university these days. We talk about this a lot on the podcast because of the extraordinary cost of student loans and the handicap that it gives you once you leave for actively paying nine percentage points of extra income tax every year, etc. and the question now is, are you learning anything valuable at university? And I hear more and more from students at their exams or at home, open book, 24 hours, etc. and a lot of them will then use I for that, I'm sure. At which it sounds like it makes the entire university experience not just pointless, but worse than pointless.”
The next generation of professionals will need a different set of cognitive abilities to use AI effectively than current generations needed: the ability to orchestrate AI tools, break down complex tasks into sub-questions that AI can answer, recompose the AI outputs, and maintain fundamental understanding of the domain—these are different from but complementary to foundational domain expertise.
“a deep understanding of how they work and ability to, you know, their own set of, of cognitive skills which will be necessary for the next generation, which which weren't necessary for our generation, you know, the the ability to orchestrate these tools, the ability to break down tasks and, and in such a way that you can use these tools to answer the sub questions and, and recompose that. And for me, you know this, there's a whole set of different cognitive abilities which are necessary to use these tools effectively.”
Looking back at previous technological waves like social media, the next generation that grew up with those tools became very smart about how to use them and didn't encounter the same faux pas problems, suggesting digital natives may use AI better than those who adopted it later.
“I'd say the my my point of optimism on this is, you know, I think if you look back to some of these previous technological waves, you know, if you take social media, you know, if again, from, from, for, for our generation, you know, we were hopeless users of those systems initially and people made all sorts of faux pas, etc. but then the next generation who had grown up with those tools, you know, were very, very smart at how to use them and didn't encounter the same sort of set of issues. They were native. They were digitally native.”
It's very shortsighted if you allow the current generation of experienced professionals to get a huge productivity bump from AI tools but don't think about how you replace them or how the next generation of professionals will actually operate.
“But I, I just think that is a very dangerous path to go down, you know? So, um, I, I think there's real value in having humans learning to do these, these tasks, you know, which have white collar profession, you know, and having them do it without access to these systems. Um, and so that you, you, you then retain that ability to, you know, oversee, override these systems when, when they're not producing the right answers. Um, but you have to invest in the human capital first. Um, so I guess the, the analogy would be with pilots.”
The question of how AI will reshape civilization depends on whether we passively and unthinkingly let change happen or whether we have a debate about how these tools are used, where they're used, and whether we proactively seek to control it rather than passively accepting our fate.
“I think that, um, I think that it does bring change. And the question to my mind is, are we going to passively and unthinkingly let this change happen? Or do you have a debate about, you know, how these tools are used, where they used, um, you know, do do we passively accept our fate with this, or do we even proactively seek to control it?”
The Dunning-Kruger effect is broken by AI use because normal learning involves becoming increasingly aware of how little you know as you put more effort into a topic, but AI use completely bypasses that metacognitive understanding of the limitations of your own knowledge.
“It's it's because you you I think it's called the Dunning-Kruger effect. This sort of, um, this, you know, as, as you put more effort into something, you know, if you, if you, um, um, you, you don't know what you don't know effectively. And as you put more and more effort into it, you become more aware of how little you know of the topic. But you completely break that effect, that understanding of the limitations of your own knowledge. When you when you start using these tools and that's that's the real weakness, the real interruption of the learning process.”
Elon Musk buying Twitter makes economic sense because from the perspective of controlling cultural evolution vectors (variation, transmission, selection), controlling the transmission mechanism for ideas is a valuable asset, making a media platform more economically valuable than surface-level market valuation suggests.
“And you say as well that you now understand why Elon Musk bought Twitter. At the time, it was challenging to understand the economic rationale... Um, but but when you think of it in terms of the impact that it has on the cultural evolution and shaping ideas. You can see why that would be a valuable asset to somebody.”
During my 30 years in Edinburgh, I used to be able to visualize places in my head and navigate there without a satnav, but having turned the satnav on every time I go somewhere, I can't do that anymore—it's not that I lost the skill suddenly, it's neglect where each time I didn't make the effort, that piece of my brain became less utilized and shrank.
“you know, I used to be able to I've lived in Edinburgh for 30 years, and, you know, I used to be the case that I could think, right, I wanted to go to this place, and I, and I visualize where I was going in my head and I'd set off. Um, but just because I turn the satnav on every time I go somewhere in the car now, and it's got to the point where I actually can't do that anymore, and it's I haven't suddenly lost that skill, but it's neglect and and it's sort of a each time I've not made that effort that that piece of my brain has become less utilized, has shrunk, and now I can't do it.”
The narrower investment issue is balancing driving productivity with ensuring capabilities for the future, but Tom doesn't extend beyond this because gains today are coming at the expense of future capability, making the trade-off irrelevant from a hard-nosed investment perspective.
“the sort of narrower investment standpoint, um, you know, it is this is this really tricky balance between, um, driving productivity and, uh, making sure you have the capabilities for the future. Um, I, you know, and I, I wouldn't extend it beyond that because I, you know, in a way, it's the it's the inverse. It's the, the, you know, the gains that people are making today are coming at the expense of, of people. So that's you know, it. It's not something that, you know, from a, from just my cold investment, a hard nosed sort of view of the world, the, you know, you can really take into account.”
Tom has no specific investment implications from his AI-cognition thesis—he studied it out of personal interest in how it affects him and his children, not to generate portfolio recommendations.
“Honestly I don't. I don't really think this that there is a particular investment implication from this. You know, I went into it because it's something I was interested in and I was interested in how it was affecting me. I was thinking about, you know, what it means for my kids.”