The Agent Loop: How AI Coding Agents Actually Work Under the Hood

Strip away the hype and an AI agent is a simple loop. Here is what each step does and why the loop matters more than the model.

Hafeez Baig•Nov 18, 2025•9 min read

Introduction

The word "agent" has been stretched until it means very little. It is applied to autocomplete, to chat interfaces, and to genuinely autonomous systems, often on the same product page. Many developers can use an agent without being able to say what one is.

The underlying structure is simpler than the marketing suggests. An AI agent is a loop. A model proposes an action, ordinary code executes that action, the result is appended to the conversation, and the loop runs again until a stopping condition is met.

That structure is worth understanding precisely, because it explains nearly every behaviour you will observe in practice. It tells you why an agent gets stuck, why costs climb faster than expected, and why a long run drifts away from the task it was given. This post builds that model from the ground up.

1. The Loop in Plain Words

An agent cycles through four steps. The names vary between authors, but the shape is consistent.

Perceive. The agent reads its current state. In practice this means reading the conversation so far: the original goal, plus everything that has happened since.
Think. The model decides what to do next. It produces either a final answer or a request to use a tool.
Act. Your code, not the model, executes that tool. It reads the file, runs the test, calls the API.
Observe. The result of the tool is appended to the conversation. The agent now knows something it did not know a step earlier.

Then the cycle repeats.

Note which step the model actually owns: only the second. The model is a text generator that emits a proposal. It cannot touch your filesystem. It can state that it would like to read config.json, and nothing happens unless the surrounding program acts on that request.

The model supplies judgement. The loop supplies capability. Neither accomplishes much alone.

2. Tools Are What Make an Agent an Agent

Remove the tools and what remains is a chatbot reasoning in isolation. Tools are the entire distinction.

A tool is a function you expose to the model, described in terms the model can read. A description has three parts:

A name, such as read_file.
A description of when to use it, such as "Read the contents of a file from disk".
A schema for the inputs, such as { "path": "string" }.

You pass that list to the model alongside the user's request. The model replies with structured output naming the tool it wants and the arguments to use. Your code parses that output, runs the real function, and returns the result to the conversation.

This is the point most often misunderstood. The model does not possess tools. It possesses descriptions of tools, and produces text shaped like a request. The capability lives entirely in your code. If you never implemented read_file, the model can request it indefinitely and nothing will occur.

That is also the security model, stated compactly: an agent can only do what you have written a function for. The tool list defines the blast radius.

3. A Minimal Agent Loop in Python

The complete idea fits in roughly thirty lines. It is worth reading closely, because the rest of this article refers back to it.

python

def run_agent(goal, tools, max_steps=15):
    # The conversation IS the memory. Nothing else persists.
    messages = [
        {"role": "system", "content": "You are a coding agent. Use tools to reach the goal."},
        {"role": "user", "content": goal},
    ]

    for step in range(max_steps):
        # THINK: ask the model what to do next
        reply = model.generate(messages=messages, tools=tools)
        messages.append(reply)

        # TERMINATE: no tool requested means the model considers itself done
        if not reply.tool_calls:
            return reply.content

        # ACT + OBSERVE: run each requested tool, feed results back
        for call in reply.tool_calls:
            try:
                result = tools[call.name](**call.arguments)
            except Exception as e:
                result = f"Error: {e}"   # errors are observations too

            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result),
            })

    return "Stopped: hit the step limit without finishing."

This is a real agent, not a simplified illustration of one. Production systems add retries, streaming, permission checks, and better bookkeeping, but that skeleton is what they wrap.

Observe what messages does. It begins with two entries and grows on every iteration. That list is the agent's entire mind. Section 6 examines what happens as it expands.

4. The System Prompt and Context Window as Working Memory

A language model retains nothing between calls. Each call to model.generate starts fresh, and the model sees only what was passed in.

An agent remembers a file it read three steps ago only because you sent it again. The messages array is resubmitted in full on every iteration. Memory is not a property of the model. It is something your loop performs by resending history.

Two components matter here:

The system prompt is the standing instruction at position zero. It carries the agent's rules: which tools exist, how to behave when uncertain, when to stop. It never scrolls out of view.
The context window is the hard limit on how much text the model can read at once, measured in tokens. A token is roughly three quarters of a word. When history outgrows the window, something must be removed.

This yields a useful reframing:

The agent's working memory is a whiteboard that is rewritten and reread in full at every step. Whatever is not on the board does not exist.

Good agents are, to a large degree, good context management: deciding what goes on the board, what gets summarised, and what is discarded.

5. How the Loop Terminates

A loop without a stopping condition is a defect rather than an agent. There are three sound ways to terminate, and a serious implementation uses all of them.

Stop condition	What triggers it	What it means
Goal reached	Model returns a normal answer with no tool call	The intended path. The agent considers the work complete.
Max iterations	Step counter is exhausted	The safety net. Prevents indefinite spinning.
Error budget	Too many consecutive failed tool calls	Something is structurally broken. Stop consuming budget.

The first condition deserves a caveat. "The agent considers the work complete" is not equivalent to the work being complete. The model has merely stopped producing tool calls. Nothing has verified the outcome.

For genuine confidence, make verification a tool. Run the test suite or the linter, and let its output be the evidence that permits the loop to exit. A passing test is a fact. A model asserting that the task is finished is only a claim.

The error budget is the condition most often omitted. Track consecutive failures separately from total steps. An agent that has failed the same command five times running is unlikely to succeed on the sixth.

6. Why Context Accumulation Causes Drift and Cost Growth

Return to the code. Every iteration appends. Nothing is ever removed. This produces two problems that tend to arrive together.

Cost grows faster than the step count. You pay per token, and the full history is resent on every step. Step one sends a short history. Step twenty sends everything from the preceding nineteen. Across twenty calls you have not paid twenty times the first call; you have paid closer to the sum of a growing series. If each step appends a chunk of file contents, then by step thirty the agent is rereading all of it, every time, to make a single decision. Long runs are expensive for structural reasons, not because the model is costly.

Attention drifts. The goal is one sentence at position zero. By step twenty it sits beneath thousands of tokens of stack traces, file dumps, and abandoned attempts. The signal to noise ratio of the context collapses, and the agent begins optimising for the most recent error rather than the original request. It fixes the test it broke while fixing the test it broke while implementing the feature, and the feature is never delivered.

Measures that help:

Return small tool results. Do not return a two thousand line file when the model needs one function. Truncate, and say that you truncated.
Summarise older turns. Replace early steps with a short account of what was established.
Restate the goal. Periodically reintroduce the objective into recent context.
Prefer fewer, sharper steps. Every step becomes memory you pay for later.

7. Chatbot Versus Agent: One Turn or Many

A chatbot is a single turn. You ask, it answers, the exchange ends. The quality of the answer depends entirely on what the model already knew. It cannot check anything. If it is wrong, it is wrong confidently, and nothing in the system will detect it.

An agent is many turns with feedback. It can be wrong at step three, read the error message at step four, and correct itself at step five. The loop converts a single guess into an iterative search.

That feedback edge is the core of the value proposition. It explains why an agent with a modest model and a good test suite frequently outperforms a stronger model with no tools. The modest one gets to discover that it is wrong.

8. Common Failure Modes

These are worth naming, because a named failure is easier to correct.

Looping without progress. The agent repeats the same failing command, or alternates between two fixes that undo one another. The cause is that no new information is entering the loop, so the model keeps reaching the same conclusion from the same context. Cap the step count, detect repeated identical calls, and inject a correction such as "that approach has failed twice, try a different one".

Hallucinated tool calls. The agent calls deploy_to_prod, which you never implemented, or passes {"filename": ...} when the schema specifies {"path": ...}. The cause is that the model generates plausible text, and a tool that ought to exist is highly plausible. Validate every call against the schema before execution, and return the validation error as an observation. An agent can usually recover if told precisely what was wrong.

Losing the goal. Twenty steps in, the agent is deep in a problem nobody raised. The cause is the context drift described in section 6. Restate the goal and keep runs short. A task requiring fifty steps was probably several tasks.

Unverified success. The agent declares completion without confirming anything. The cause is allowing the model to judge its own work. Make verification a tool call whose output enters the context.

Conclusion

The loop is the product. That is the point worth retaining.

Model quality matters, but you do not control it, and it improves on someone else's schedule. What you do control is everything surrounding the model: which tools you expose, how compact their outputs are, what the system prompt establishes, when the loop halts, and what evidence must exist before halting is permitted. Those decisions determine most of the difference between an agent that works and one that merely appears to.

For a first agent, I would start deliberately small: two tools, a ten step cap, one clear goal, and every message printed to the terminal so the context can be watched as it grows. Then run it against a real task.

A single agent visibly losing the thread in front of you will teach more than any amount of reading on the subject, this article included.

Written for developers building with AI agents. More on how these systems behave in practice to follow.