Why Your AI Agent Needs a Test Loop, Not a Bigger Prompt

When an agent keeps getting it wrong, the instinct is to add more instructions. A feedback signal beats a longer prompt almost every time.

Hafeez Baig•Dec 2, 2025•8 min read

Introduction

There is a familiar pattern in agent development. The agent produces code that does not compile. You open the system prompt and add a line: "Always make sure the code compiles." It fails again. You add another line about checking imports. Then another. Within a few weeks the prompt is several hundred lines of accumulated corrections, and the agent still forgets the imports.

The argument of this post is that most agent quality problems are not prompt problems. They are feedback problems. The agent is usually not failing because it does not know the rule. It is failing because nothing ever tells it that the rule was broken. A prompt states an intention. A test loop produces a fact.

One definition first. An agent here means a language model that takes actions in a loop: writes a file, runs a command, calls an API, then decides what to do next based on the result. The loop is what separates an agent from a chatbot.

1. Why the System Prompt Keeps Growing

Prompt growth is rational in the moment. The agent did something wrong, you know what correct looks like, so you write it down. It is the cheapest available fix: no code, no infrastructure, just text.

The approach degrades for three reasons.

Instructions get diluted. Model attention is a finite budget. With four rules in a prompt, each one carries weight. With sixty, each competes with the other fifty nine. The rule added last week to patch a bug now dilutes the rule added in month one that actually mattered. The additions do not stack. They trade against each other.

The model cannot verify itself. This is the central issue. The prompt says the code must compile. The model has no compiler. It is producing text that resembles code which compiles, based on patterns in its training data. Asking it to guarantee compilation is like asking someone to proofread a document in a language they only partly read. They will catch some errors and confidently miss others.

The fix is a guess. You observed one failure, inferred a cause, and wrote a rule. Neither the cause nor the fix was ever confirmed. The next run looks acceptable, so the rule appears to have worked. It may have. The model may also have gotten lucky. Without a check there is no way to tell, so the prompt fills with rules that do nothing alongside rules that actively interfere, indistinguishable from the outside.

A reliable signal that this has happened: rules that contradict each other. "Be concise" three sections above "explain your reasoning in detail." That is no longer a specification. It is a changelog of past incidents.

2. What a Test Loop Is

The structure is straightforward. Four steps:

The agent acts. It writes code, produces JSON, edits a config.
Something objective checks the result. A test suite, a compiler, a schema validator.
On failure, the error is fed back to the agent as specific text.
The agent retries with that information available.

No new model, no fine tuning, no prompt engineering. The loop simply closes the gap between action and consequence.

The mindset shift is worth stating directly. You stop trying to make the agent correct on the first attempt, and start making it cheap to be wrong. A model that fails and receives a precise error message will typically correct itself within a round or two. A model that fails silently will fail indefinitely.

3. Why an Objective Checker Beats Self-Critique

A common pattern asks the model to review its own work: "check your answer carefully and rate it out of ten." This occasionally helps. Usually it does not.

The reason is structural. An LLM asked whether it did well can be talked out of a failure, including by itself. It is generating text that resembles a review, and reviews of completed work tend to read like reviews of work completed successfully. The model will rate code that does not parse at eight out of ten.

A compiler cannot be talked out of a failure. Neither can a test suite, a linter, or a schema validator. They are indifferent to how confident the surrounding explanation was. They return an exit code.

Checker	Can it be argued with?	What it returns
LLM self-critique	Yes, easily	A plausible opinion
LLM judge (separate model)	Somewhat	A better opinion, still an opinion
Linter	No	Exact rule and line number
Type checker or compiler	No	Exact type mismatch
Test suite	No	Expected versus actual
Schema validator	No	Which field, which constraint

LLM judges are not useless. For qualities such as tone or helpfulness, a model may be the only checker available. But the deterministic checker should be the default, with a model as fallback only when the property genuinely cannot be expressed in code. It usually can. Whether a string is valid JSON is not a matter of judgement.

4. A Loop That Runs Tests and Feeds Failures Back

The implementation is intentionally unremarkable.

python

MAX_ATTEMPTS = 4

def run_with_test_loop(task):
    messages = [{"role": "user", "content": task}]
    last_error = None

    for attempt in range(MAX_ATTEMPTS):
        code = agent.generate(messages)
        write_file("solution.py", code)

        result = run_tests()  # returns (passed: bool, output: str)
        if result.passed:
            return code, attempt + 1

        # Feed back the actual failure, not an instruction to try harder.
        last_error = result.output
        messages.append({"role": "assistant", "content": code})
        messages.append({
            "role": "user",
            "content": (
                "The tests failed. Here is the exact output:\n\n"
                f"{truncate(last_error, 2000)}\n\n"
                "Fix the code so these tests pass. "
                "Do not change the tests."
            ),
        })

    raise AgentFailed(
        f"Gave up after {MAX_ATTEMPTS} attempts. Last error:\n{last_error}"
    )

Note what the feedback message does not contain. There is no instruction to be more careful. That is a prompt fix and it carries no information. The message pastes the real output, because the real output is the information.

Note also the line Do not change the tests. An agent given a failing test and the ability to delete it has been taught that the shortest path to a green result is removal of the check. The checker must sit outside the agent's reach, enforced by permissions rather than by request.

5. Designing the Feedback Signal

A loop is only as good as its checker. Three properties matter.

Speed. The checker runs several times per task. A checker that takes ninety seconds costs six minutes across four attempts, and it will quietly stop being used. Run the narrow relevant check rather than the full suite. Fast and narrow beats slow and complete.

Determinism. Same input, same verdict. A flaky test inside a feedback loop is worse than no test. The agent is penalised for correct work, then modifies code that was already sound, and the loop begins to circle.

Specificity. This is where most implementations lose their value. Compare:

Error: validation failed

with:

Error: field 'user.created_at' expected ISO 8601 string, got integer 1733097600 (line 14)

Both are equivalent to the calling program. To the model, the first is noise and the second is a repair instruction. Where a checker emits vague errors, improving those messages is often the highest value change available, and it improves the experience for the engineers reading them as well.

6. Retry Budgets and Knowing When to Stop

A loop needs a stopping rule, otherwise an agent will consume tokens indefinitely on an impossible task.

Cap the attempts. Three or four is a reasonable starting point. Tasks that have not converged by then rarely converge later.
Stop on repeated errors. If attempt three reproduces the error from attempt two, the agent is stuck rather than progressing. Break early.
Fail loudly. When the budget is exhausted, surface the last real error to a human. Do not return the broken output as though it succeeded. A loop that conceals failures is worse than no loop, because it has earned trust it does not merit.
Track the attempt count. If the average creeps from roughly one attempt toward three, something upstream has changed. That number is a useful health metric.

7. The Parallel With Test Driven Development

The structure should look familiar. Test driven development prescribes writing the test first, watching it fail, then writing code until it passes.

Substitute the agent for the developer and the loop is identical. Define success as something a machine can check, allow the work to be wrong initially, and let the signal drive it toward correct.

The difference is instructive. TDD was always partly about discipline, forcing developers to define correctness before becoming absorbed in implementation. Agents do not need discipline. They need a gradient, and the test loop supplies one. The practice arguably suits machines better than it ever suited people.

This reframes the opening question of an agent project. The useful question is not what to tell the model. It is how you will know whether it worked. Without an answer, no prompt will compensate.

8. When a Bigger Prompt Genuinely Is the Answer

None of this is an argument against prompts. Sometimes the prompt is the problem. The distinction worth drawing is this.

Missing context is a prompt problem. The agent does not know the table is named users_v2. It does not know the team has banned a library. It does not know what "the pipeline" refers to in your codebase. No feedback loop will invent these facts. The agent cannot deduce information it was never given. Write it down.

Missing feedback is a loop problem. The agent knows the rule and breaks it regardless. It is inconsistent: correct on one run, wrong on the next, with the same input. It produces output that appears correct and is not. Additional words will not address any of that.

The diagnostic is simple. Ask whether a competent new colleague, given only this prompt and no other knowledge, could complete the task. If not, add context. If yes, and the agent still fails, the problem is verification, and the remedy is a checker.

Conclusion

Adding one more line to the system prompt is an understandable reflex. It costs nothing and resembles progress. It also optimises the wrong variable, attempting to make the model correct through instruction alone.

The alternative is less interesting to build. Write a checker. Make it fast, deterministic, and specific about what broke. Feed the failures back. Cap the retries. Fail loudly when the budget is exhausted.

A useful side effect follows. The system prompt gets shorter. The defensive rules accumulated over months become unnecessary, because the loop enforces what the prose could only request.

For anyone currently looking at an overgrown prompt, the first step is not a rewrite. Take the failure that costs you the most, write the smallest possible check for it, and let the agent run into it.

An agent that can fail safely will improve. An agent that can only be instructed will not.

Written for engineers building agent systems intended to run in production.