
What this covers
This IBM Technology explainer, presented by Maya Murad (the transcript itself lists only an unidentified speaker), starts from the limits of standalone models and works toward agents. A model is bound by its training data and is costly to retune. A single model also cannot answer a question like "how many vacation days do I have left?", because it has no access to personal records and does not know who is asking. The video's first answer is compound AI systems: modular designs that wrap a model with databases, tools, output verifiers, and programs that decompose queries. It argues these are easier to design and faster to adapt than tuning a model.
The middle of the video turns on who decides the path through those components. In conventional systems, including retrieval augmented generation (RAG)), the human designer fixes the control logic. Those systems fail when a query falls outside their intended scope, such as a weather question put to a vacation-days assistant. The alternative is an agentic approach, where the language model controls the logic. The video attributes this shift to recent gains in LLM reasoning, and it names three capabilities: reasoning and planning, acting through tools, and memory, covering both stored reasoning logs and conversation history. A ReACT agent is the worked pattern: plan, call a tool, observe the result, and iterate until the answer is reached. A sunscreen-planning example chains memory lookup, weather forecasts, public health guidance, and arithmetic.
The video ends on a tradeoff rather than a recommendation to use agents everywhere. For narrow, predictable tasks, programmatic control is more efficient, since agents would add needless looping. For varied, open-ended work such as resolving GitHub issues, configuring every path by hand is impractical, so agentic control pays off. It frames this as a sliding scale of autonomy and predicts that a human will usually stay in the loop while accuracy improves.
AI agents represent a fundamental shift from monolithic models to compound AI systems where large language models control program logic, enabling systems to reason through complex problems and dynamically select tools—a paradigm that trades off efficiency for flexibility depending on task complexity.
- Monolithic models are limited by training data and hard to adapt; compound AI systems modularize components (models, tools, verifiers) to solve problems more flexibly
- LLM agents use reasoning and tool-calling controlled by the model itself rather than fixed programmatic logic, enabling dynamic problem decomposition
- The tradeoff between programmatic control (efficient for narrow, well-defined tasks) and agentic control (necessary for complex, varied problems) requires intentional system design choices
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
The magic of compound AI systems is unlocked when you build systems around the model and integrate the model into existing processes, providing it with access to relevant databases and tools.
“So the magic gets unlocked when I start building systems around the model and actually take the model and integrate them into the existing processes I have.”
Compound AI systems recognize that certain problems are better solved by applying principles of system design, which treats the solution as modular with multiple interchangeable components (models, tools, verifiers) rather than tuning a single model.
“This is an example of a compound AI system, and it recognizes that certain problems are better solved when you apply the principles of system design. So what does that mean? By the term 'system', you can understand there's multiple components. So systems are inherently modular. I can have a model, I can choose between tuned models, large language models, image generation models, but also I have programmatic components that can come around it.”
System approaches are inherently easier to design and faster to adapt than tuning a model, because you can break down what you want your program to do and pick the right components to solve it.
“When we talking about a system approaches, I can break down what I desire my program to do and pick the right components to be able to solve that. And this is inherently easier to solve for than tuning a model. So that makes this much faster and quicker to adapt.”
Compound AI systems traditionally have programmatic control logic that is defined by the human designer, but an alternative approach is to put a large language model in charge of the control logic.
“compound AI systems, we said most of them have programmatic control logic. So that was something that I defined myself as the human. Now let's talk about, where do agents come in? One other way of controlling the logic of a compound AI system is to put a large language model in charge”
The third capability of LLM agents is the ability to access memory, which includes inner logs of the model's reasoning process (thinking out loud) that can be stored and retrieved at different points, as well as conversation history that enables personalization.
“Third capability, that is the ability to access memory. And the term 'memory' can mean a couple of things. So we talked about the models thinking through the program kind of how you think out loud when you're trying to solve through a problem. So those inner logs can be stored and can be useful to retrieve at different points in time. But also this could be the history of conversations that you as a human had when interacting with the agent. And that would allow to make the experience much more personalized.”
A standalone language model cannot answer a personal query about vacation days because it lacks access to sensitive personal information and doesn't know the identity of the user.
“I think we can all expect that this answer will be incorrect, because the model doesn't know who I am and does not have access to this sensitive information about me.”
There is a sliding scale of AI autonomy, and system designers must examine the tradeoffs they want in terms of autonomy for specific problems.
“The way I like to think about it is you have a sliding scale of AI autonomy. And you would the person defining the system would examine what trade offs they want in terms of autonomy in the system for certain problems”
Models on their own are limited by the data they've been trained on, which impacts what they know about the world and what tasks they can solve.
“models on their own are limited by the data they've been trained on. So that impacts what they know about the world and what sort of tasks they can solve.”
Models are hard to adapt; tuning a model would take an investment in data and resources.
“They are also hard to adapt. So you could tune a model, but it would take an investment in data, and in resources.”
The key power of agent systems is that there are many possible paths to solve a complex problem, making the system highly modular and capable of handling much more complex problems than fixed-path systems.
“So what's really powerful here is there's so many paths that can be explored in order to solve a problem. So this makes the system quite modular. And I can hit it with much more complex problems.”
For narrow, well-defined problems with a predictable scope (e.g., vacation queries where weather questions are unexpected), programmatic control is more efficient than agentic control because every query will be answered the same way and agents would cause unnecessary looping and iteration.
“So for especially problems that are narrow, well-defined. So you don't expect someone to ask them about the weather when they need to ask about vacations. So a narrow problem set. You can define a narrow system like this one. It's more efficient to go the programmatic route because every single query will be answered the same way. If I were to apply the genetic approach here, there might be unnecessarily looping and iteration.”
For complex tasks that involve a variety of queries and diverse problem types (e.g., solving GitHub issues independently), an agentic approach is helpful because it would take too much human effort to configure every single path in a programmatic system.
“But if I expect to have a system, accomplish very complex tasks like, say, trying to solve GitHub issues independently, and handle a variety of queries, a spectrum of queries. This is where an agent de Groot can be helpful, because it would take you too much effort to configure every single path in the system.”
When given a complex question, providing the first answer that comes to mind is likely to be wrong, but breaking down the problem, understanding where external help is needed, and taking time to solve it increases the chances of success.
“So I might give you a complex question, and if you would just give me the first answer that pops into your head, very likely the answer might be wrong, but you have higher chances of success if you break it down, understand where you need external help to solve some parts of the problem, and maybe take an afternoon to solve it.”
In a compound system designed to answer the vacation days query, the model should be prompted to generate a search query, which is sent to the database to fetch the information, and the model then generates a natural language response.
“the model would be prompted to create a search query, and that would be a search query that can go into the database that I have. So that would go and fetch the information from the database, output an answer, and then that would go back into the model that can generate a sentence to answer, so, 'Maya, you have ten days left in your vacation database.'”
RAG systems and typical compound AI systems are defined by fixed programmatic control logic where a specific path must be followed to answer queries, causing them to fail when asked about topics outside their designed scope (e.g., asking about weather when designed for vacation queries).
“Most RAG systems and the example I use below are defined in a certain way. So if I bring a very different query, let's ask about the weather in this example here. It's going to fail because this the path that this program has to follow is to always search my vacation policy database. And that has nothing to do with the weather.”
In the sunscreen planning example, an agent solves a complex multi-step problem by: (1) retrieving vacation days from memory, (2) looking up weather forecasts to determine sun hours, (3) accessing public health websites for sunscreen dosage recommendations, and (4) performing math to calculate bottle quantities.
“One is how many vacation days am I planning to take? And maybe that is information the system can retrieve from its memory. Because I asked that question before. Two is how many hours do I plan to be in the sun? I said, I plan to be in there a lot, so maybe that would mean looking into the weather forecast, for next month in Florida and seeing what is the average sun hours that are expected. Three is trying maybe going to a public health website to understand what is the recommended dosage of sunscreen per hour in the sun. And then for doing some math, to be able to determine how much of that sunscreen fits into two ounce bottles.”
In a ReACT agent, the user query is fed into a model with instructions to think slow and plan before acting; the agent can then decide whether to use external tools, observe the results, evaluate whether they answer the question, and iterate on the plan if needed until reaching a final answer.
“What happens when I configure a REACT agent? You have your user query that gets fed into a model. So an alarm the alarm is given a prompt. So the instructions that's given is don't give me the first answer that pops to you. Think slow planning your work. And then try to execute something. Tried to act. And when you want to act, you can define whether. If you want to use external tools to help you come up with the solution. Once you get you call a tool and you get an answer. Maybe it gave you the wrong answer or it came up with an error. You can observe that. So the alarm would observe. The answer would determine if it does answer the question at hand, or whether it needs to iterate on the plan and tackle it differently. Up until I get to a final answer.”
Compound AI systems can include output verifiers, programs that break down queries to increase correctness, database searches, and various tools that can be combined to solve specific problems.
“I can have output verifiers. I can have programs that can that can take a query and then break it down to increase the chances of the answer being correct. I can combine that with searching databases. I can combine that with different tools.”
The second capability of LLM agents is the ability to act: external programs called tools that the model can define when and how to call in order to execute the solution to the question.
“Another capability of agents is the ability to act. And this is done by external programs that are known in the industry as tools. So tools are external pieces of the program, and the model can define when to call them and how to call them in order to best execute the solution to the question they've been asked.”
The term 'control logic' in compound AI systems refers to the path or sequence of steps a program must follow to answer a query.
“So when we say the path to answer a query, we are talking about something called the control logic of a program.”
When a large language model is put in charge of control logic in an AI system, this is called an agentic approach.
“And when we put a LLMs in charge of the logic, this is when we're talking about an agentic approach.”
The first capability of LLM agents is the ability to reason: the model is prompted to come up with a plan and reason about each step of the process.
“Let's break down the components of LLM agents. The first capability is the ability to reason, which we talked about. So this is putting the model at the core of how problems are being solved. The model will be prompted to come up with a plan and to reason about each step of the process along the way.”
In most cases, there will continue to be a human in the loop as accuracy is improving in agent systems.
“And of course, you will have a human in the loop in most cases as the accuracy is improving.”
This shift to LLM-controlled logic is only possible because large language models have recently demonstrated tremendous improvements in reasoning capabilities, allowing them to break down complex problems and create plans to tackle them.
“and this is only possible because we're seeing tremendous improvements in the capabilities of reasoning of large language models. So large language models, you can feed them complex problems and you can prompt them to break them down and come up with a plan on how to tackle it.”
There are two ends of a spectrum for controlling system logic: 'think fast' systems that act as programmed without deviation (programmatic control), and 'think slow' systems that create a plan, attack each part, detect where they get stuck, and readjust the plan as needed (agentic control).
“Another way to think about it is, on one end of the spectrum, I'm telling my system to think fast, act as programmed, and don't deviate from the instructions I've given you. And on the other end of the spectrum, you're designing your system to think slow. So, create a plan, attack each part of the plan, see where you get stuck, see if you need to readjust the plan.”
Tools in agent systems are external pieces of programs including web search, database search, calculators, code execution, other language models for specific tasks (like translation), and APIs—basically any external program the model should have access to.
“So an example of a tool can be search, searching the web, searching a database at their disposal. Another example can be a calculator to do some math. This could be a piece of program code that maybe might manipulate the database. This can also be another language model that maybe you're trying to do a translation task, and you want a model that can be able to do that. And there's so many other possibilities of what can do here. So these can be APIs. Basically any piece of external program you want to give your model access to.”
Retrieval Augmented Generation (RAG) is one of the most popular and commonly used compound AI systems.
“You also might be popular with retrieval augmented generation (RAG), which is one of the most popular and commonly used compound AI systems out there.”
One of the most popular ways of configuring agents is through ReACT, which combines the reasoning and act components of LLM agents.
“So the way of configuring agents, there's many are ways to approach it. One of the more most popular ways of going about it is through something called ReACT, which, as you can tell by the name, combines the reasoning and act components of LLM agents.”
The field is still in the early days of agent systems, with rapid progress being driven by the combination of system design principles and agentic behavior.
“And we're still in the early days of agent systems. We're seeing rapid progress when you combine the effects of system design with a genetic behavior.”
Compound AI systems are here to stay, and the major trend for 2024 is that they will become more agentic, with an increasing amount of autonomy being placed in the LLM to control the system logic.
“So going back to the concept of compound AI systems, compound AI systems are here to stay. What we're going to observe this year is that they're going to become more agent tech.”