How AI agents chain tools, memory & reasoning
An LLM in a loop that plans, acts, observes — and keeps going until it's done.
An agent wraps an LLM in a loop: it plans a step, calls a tool, observes the result, and repeats until the goal is met. The core pieces are the LLM for reasoning, memory (short-term context plus a long-term store), tools via function-calling, and a planner that orchestrates the cycle.
Receive the goal
A high-level goal enters the loop — 'book me a flight under $400' or 'find and fix the failing test'. The LLM sits at the center as the reasoning engine, and it's told up front which tools it has and what memory it can draw on. Unlike a single chat completion, the agent isn't expected to answer in one shot; it's expected to work the problem.
Reason and plan
The LLM inspects the goal alongside its current context, then reasons about the single next concrete step to take. It pulls in short-term memory (what's happened so far this run) and can query long-term memory for durable facts or past experience. This 'think before acting' step is what the ReAct pattern makes explicit, and it's why agents can handle multi-step problems rather than just reacting blindly.
Call a tool
The plan resolves into an action: the model emits a structured function-call naming a tool and its arguments — search the web, run code, hit an API, query a database. The orchestrator intercepts that call, actually executes the tool, and this is the moment the agent affects the world beyond its own text. Tools are what let a language model do things instead of merely describe them.
Observe and repeat
The tool's output — a search result, an error message, an API response — flows back into the model's context as an observation. The model reads it, updates its understanding, and decides whether the goal is met or another step is needed. If it isn't done, the whole perceive-reason-act-observe cycle runs again, each pass building on what the last one learned.
Finish
Once the accumulated observations satisfy the goal, the model stops calling tools and produces its final answer. A well-built agent also enforces guardrails here — a step limit or budget — so it terminates even if it can't fully succeed. The output is the product of many reasoning and tool steps, not a single generation.
From a single answer to an agent
A plain LLM call is a function: text in, text out. You give it a prompt, it returns one completion, and the interaction is over. That is enormously useful, but it has a hard ceiling — the model can only reason with what's already in the prompt, it can't take any action in the world, and it gets exactly one attempt. Ask it to "check today's exchange rate and convert this invoice" and it can't, because it has no way to look anything up; it will guess.
An agent removes that ceiling by wrapping the LLM in a loop and giving it tools. Instead of answering in one shot, the agent works toward a goal over multiple steps: it thinks about what to do, does something in the real world, sees what happened, and thinks again — repeating until the job is done. The model stops being an oracle you query and becomes a reasoning engine that drives a process.
The clearest way to hold this in your head is a capable assistant with a goal, a notepad, and a phone. They consider the next move, make a call or look something up, jot down what came back, and repeat until finished. That behavior — autonomous, multi-step, tool-using, goal-directed — is what distinguishes an agent from a single completion. The defining feature is not a bigger model or a cleverer prompt; it is the *loop*. Everything else in this article is the machinery that makes the loop work: the reasoning core, memory, tools, and the orchestrator that ties them together.
The core loop: perceive, reason, act, observe
The engine of every agent is a cycle, most commonly formalized as ReAct (Reason + Act). It runs like this, over and over:
- Perceive / plan. The model looks at the goal and everything that's happened so far, and reasons about the single best next step. Crucially, this thinking is done in text — the model literally writes out its reasoning ("I need the current price, so I should search for it"), which both improves the decision and makes the agent's behavior inspectable.
- Act. That reasoning resolves into an action: a call to one of the agent's tools, with specific arguments.
- Observe. The tool runs and its result is fed back into the model's context as an observation.
- Repeat. The model reads the observation, updates its plan, and either acts again or, if the goal is met, produces a final answer.
Why does interleaving reasoning with acting matter so much? Because it lets the agent adapt. A rigid script would break the moment reality diverged from the plan — a search returns nothing, an API errors, a result contradicts an assumption. ReAct handles all of that naturally: the observation from a failed step becomes input to the next round of reasoning, so the agent can retry, choose a different tool, or change course. This closed feedback loop is what turns a one-shot predictor into something that can grind through genuinely multi-step problems — debugging code, researching a question across many sources, completing a booking — where the right next action can only be known after seeing the results of the last one.
Tools: how an agent acts on the world
An LLM on its own can only produce text. Tools (exposed via what's called function-calling or tool-calling) are what let it reach outside that box and actually do things — the difference between describing a web search and performing one.
Mechanically, the setup is elegant. You describe each available tool to the model in a structured way: a name, what it does, and a schema of its parameters (a "get_weather" tool that takes a city string, say). When the model decides to use one, it doesn't execute anything itself — it can't. Instead it emits a structured request naming the tool and filling in the arguments. The surrounding program — the orchestrator — parses that request, actually runs the corresponding real function (hitting an API, running a query, executing code), and passes the return value back into the model's context as the observation for the next loop.
This division of labor is the whole trick: the model provides the *judgment* of which tool to call and with what inputs, while ordinary, deterministic code provides the *execution*. It also means an agent's capabilities are exactly the tools you give it. Typical toolsets include web search, a code interpreter, database queries, file read/write, HTTP requests to internal services, and — increasingly — whole catalogs of tools exposed through a standard like MCP (the Model Context Protocol), which lets an agent plug into external tools without bespoke integration code for each one. Choosing, describing, and constraining these tools well is one of the highest-leverage parts of building a reliable agent.
Memory: short-term context and long-term stores
Reasoning and acting aren't enough on their own — an agent also has to *remember*, and it does so on two distinct tiers, because the two solve different problems.
Short-term (working) memory is simply the model's context window: the running transcript of the current task — the goal, each thought, each tool call, and each observation so far. This is what lets step 7 build on what step 3 discovered. But it is finite and it is temporary. A long-running task can overflow the window, forcing the agent to summarize or prune older turns, and everything in it vanishes when the task ends. It is the agent's scratchpad, not its long-term store.
Long-term memory persists knowledge across steps and across entirely separate sessions, and it typically lives outside the model — often in a vector database, retrieved on demand exactly like RAG. This is how an agent remembers a user's preferences from last week, recalls facts it learned on a previous run, or draws on a knowledge base far too large to ever fit in a prompt. When the agent needs something durable, it queries this store and pulls the relevant pieces into short-term context for the current step.
The interplay is the point: short-term memory holds the here-and-now of a single task, while long-term memory is the durable knowledge the agent selectively loads when relevant. Managing this well — deciding what to keep in context, what to write to the long-term store, and what to retrieve back — is a core design problem, because an agent that forgets what it's doing loops or repeats itself, and one that can't recall past knowledge starts every task from zero.
Orchestration, failure modes, and building reliably
Tying the loop, tools, and memory together is the orchestrator — the surrounding program (LangGraph, the OpenAI or Anthropic agent SDKs, or your own code) that runs the cycle, dispatches tool calls, injects observations back into context, tracks state, and enforces stopping conditions. Simple agents run one linear loop; more advanced systems use multi-agent designs, where an orchestrator delegates sub-tasks to specialized agents (a "researcher", a "coder", a "critic") and combines their results, trading added complexity for division of labor.
It's essential to understand where agents fail, because they fail in characteristic ways. They can get stuck in loops, repeating the same unproductive action — searching the same failed query again and again — without recognizing they're not making progress. They hallucinate tools or arguments, trying to call a function that doesn't exist or inventing a plausible-but-wrong parameter. Small errors compound: a slightly wrong observation early on poisons every subsequent reasoning step, and the agent confidently marches off in the wrong direction. And because each step is an LLM call, a long chain of steps is slow and can get expensive fast.
Reliability therefore comes from constraints, not from trusting the model. Practical guardrails include a hard cap on the number of steps and on token/dollar budget so a runaway agent always terminates; validating tool arguments against their schema and returning clear errors the agent can recover from; keeping the toolset small and well-described so the model isn't overwhelmed; adding retries and human-in-the-loop checkpoints for consequential actions; and logging the full reasoning trace so failures are debuggable. Agents are powerful precisely because they're autonomous — but that same autonomy is why the engineering discipline around the loop matters as much as the model inside it.
An agent is an LLM running in a loop that plans, acts, observes results, and repeats until a goal is met.
The LLM does the reasoning — deciding what to do next given the current state.
Memory has two tiers: short-term context in the prompt and a long-term store for durable knowledge.
Tools (function-calling) let the agent reach out to APIs, run code, or search — acting on the world.
A planner/orchestrator drives the ReAct-style cycle: perceive → reason/plan → act → observe → repeat.
Quick quiz
What makes something an 'agent' rather than a single LLM call?
What are the two tiers of an agent's memory?
What is MCP (Model Context Protocol)
One open protocol that connects AI apps to any tool or data source — USB-C for AI.
What is Retrieval-Augmented Generation (RAG)
Ground an LLM in your own data so it stops making things up.
How the Transformer architecture works
The architecture behind GPT, Claude, Gemini and Llama — one block at a time.