Loop Engineering with AI: How to Build Self-Running Agents That Ship

What Is Loop Engineering and Why Does It Matter?

Loop engineering is the shift from one-shot AI prompts to self-running systems. Instead of asking an AI to do a task and checking the result manually, you build a loop: the agent has a goal, gathers context, takes action, evaluates the outcome, and repeats until the goal is satisfied.

This matters because the most useful AI work is not a single question-and-answer. It is sustained execution over many steps. A pull request that needs to pass CI. A bug that needs reproduction, fix, and verification. A performance benchmark that needs tuning across dozens of iterations. One prompt cannot do this. A loop can.

The loop is the new unit of AI engineering.

The Four Components of a Reliable AI Loop

Every loop has the same four parts. Weakness in any one of them makes the whole loop fragile.

1. A Clear Goal

A loop without a goal is a slop cannon. The goal must be specific, measurable, and bounded. It is the exit condition. Without it, the agent will keep going forever or declare success on the wrong thing.

2. Curated Context

Context is the fuel. The agent needs the right information at the right time, not a dump of everything at the start. Context includes:

• Tool descriptions and available skills
• Current state: diffs, logs, metrics, errors
• Memories: previous decisions, failures, constraints
• External signals: CI status, user feedback, analytics events

The best loops feed context incrementally. The agent fetches what it needs, acts, observes the result, and fetches more.

3. Evaluation

Evaluation is the agent’s ability to check itself. This is the biggest difference between loops and prompting. In a loop, the agent does the verification, not the engineer.

Common evaluation methods:

• Tests — unit, integration, and regression tests
• Evals — LLM-as-judge or structured rubrics
• Metrics — latency, throughput, error rates, cost
• Playgrounds — manual spot checks for subjective quality
• CI/CD — the canonical pass/fail signal for code changes

4. The Agent

The agent is the executor. At the simple end, this is Claude Code with the /loop or /goal command. At the complex end, it is a purpose-built harness: cron-triggered agents that read product data, emit tasks to subagents, and verify outcomes.

The Five Primitives of a Production Loop

A loop is more than a single agent repeating a task. The most useful loops are built from five primitives plus a memory layer. These primitives now ship inside the tools, which is why loops are suddenly practical.

Automations Are the Heartbeat

Automations are what make a loop a loop instead of a one-off run. You define a prompt, a cadence, and an environment. The agent runs on its own, surfaces findings, and archives the runs that find nothing. Both Codex and Claude Code support this. Claude Code uses /loop for timed re-runs and /goal for run-until-done, where a separate model checks whether the stop condition is satisfied.

Worktrees Prevent Collision

Once two agents touch the same files, you need isolation. A git worktree gives each agent its own branch and working directory while sharing the same repository history. This is the same idea as two engineers working on separate branches, but for agents.

Skills Stop Goldfish Re-Explanation

A SKILL.md file is where you write down project conventions, build steps, and “we do not do this because of that incident” knowledge. The agent reads it every run, so intent is preserved across sessions. Without skills, the loop re-derives your project from scratch every cycle.

Connectors Make the Loop Real

A loop that only sees the filesystem is limited. MCP connectors let the agent read issue trackers, query databases, hit staging APIs, and post to Slack. This is the difference between an agent that suggests a fix and a loop that opens the PR, links the ticket, and pings the channel once CI is green.

Sub-agents Separate Maker from Checker

The agent that writes the code is too nice grading its own homework. A second agent with different instructions — and sometimes a different model — catches the things the first one talked itself into. The usual split is one agent explores, one implements, one verifies. This is the only reason you can walk away from a running loop.

Memory Lives Outside the Chat

The model forgets everything between sessions. The loop cannot. A markdown file, a Linear board, or a structured progress file on disk holds what was tried, what passed, and what is still open. Tomorrow’s run picks up where today’s stopped.

Real Examples of AI Loops

Here are four loops that already work in practice.

PR Babysitter

• Goal: Get a pull request to pass CI and be merge-ready.
• Context: The diff, the test suite, CI logs, and code review comments.
• Evaluation: CI passes, no unresolved review threads, and the diff is under the change threshold.
• Loop: The agent reads CI failures, fixes the code, re-runs tests, and repeats until green.

Bug Fixer

• Goal: Fix a reported bug.
• Context: The bug report, error trace, relevant files, and reproduction steps.
• Evaluation: The reproduction test passes and no existing tests break.
• Loop: The agent reproduces the bug, proposes a fix, runs tests, and iterates on failures.

Flaky Test Hunter

• Goal: Eliminate flaky tests.
• Context: CI history, retry logs, test source code, and recent changes.
• Evaluation: The test passes consistently over N consecutive runs.
• Loop: The agent identifies patterns, applies fixes, and re-runs the suite until stability is proven.

Performance Autoresearcher

• Goal: Beat a benchmark.
• Context: The system, metrics, budget, and prior experiments.
• Evaluation: The target metric improves on the benchmark suite.
• Loop: The agent generates hypotheses, applies changes, measures results, and keeps the winners.

These are not science fiction. They are small, well-scoped loops running on real systems.

What One Loop Looks Like

Here is a concrete loop shape that works across tools. It uses the same five primitives in sequence.

1. Automation runs every morning. It calls a triage skill that reads yesterday’s CI failures, open issues, and recent commits. Findings are written to a markdown file or a Linear board.

2. For each actionable finding, the loop opens a worktree. Each agent gets an isolated branch and working directory. Their edits cannot collide.

3. A sub-agent drafts the fix. It reads the skill file, the relevant code, and the failure context.

4. A second sub-agent reviews the draft. It checks the change against the project skills, existing tests, and constraints.

5. Connectors open the PR and update the ticket. The loop links the change to the original issue and pings the channel once CI passes.

6. Memory tracks what is done. A state file records what was tried, what passed, and what is still open. The next run picks up where this one stopped.

The key insight: you designed the loop once. You did not prompt any of the individual steps. The loop prompts the agents.

Why Loops Are Taking Off Now

Loops are not new. They are an expression of several real capability improvements happening at once.

The result is that agents can now complete tasks that previously required a human engineer to sit in the chair and drive every step.

How to Build Your First Loop

You do not need a custom harness to start. The simplest loop is a tight cycle inside Claude Code.

Minimal Loop with Claude Code

# Claude Code has a /loop command and automations that support this pattern.
# Start with a bounded goal and a clear evaluation signal.

Goal: Make all tests in src/services/billing pass.

Context:
- Read the current test files in src/services/billing/**
- Read the implementation files
- Check the latest CI failure logs

Loop:
1. Identify the failing test or error.
2. Propose a fix.
3. Run the affected tests.
4. If they pass, stop. If they fail, go to step 1.

Constraints:
- Do not change public API signatures.
- Keep changes under 50 lines per iteration.

This is enough to handle many real tasks. The key is the evaluation signal. Tests are ideal because they are unambiguous.

A More Structured Loop

For production use, you want a harness that tracks state, handles failures, and logs decisions.

# Pseudocode for a structured loop
from agent import Agent, Tool

class Loop:
    def __init__(self, goal: str, evaluator, agent: Agent):
        self.goal = goal
        self.evaluator = evaluator
        self.agent = agent

    def run(self):
        while True:
            context = self.agent.fetch_context()
            action = self.agent.decide(context, self.goal)
            result = self.agent.execute(action)
            score = self.evaluator.check(result)

            if score.is_done:
                return result

            self.agent.remember(result, score)

The harness is the easy part. The hard part is defining the goal, the evaluation, and the context boundaries.

Loops vs. One-Shot Prompting

One-shot prompting is not going away. It is the right tool for quick answers and small tasks. Loop engineering is the right tool for anything that requires iteration, verification, or persistence.

The Self-Driving Product Vision

The long-term promise of loops is the self-driving product. Instead of an engineer prompting an agent to fix a bug or improve a funnel, the agent monitors signals and acts on its own. The loop becomes:

1. Collect data from analytics, support tickets, and user feedback.
2. Identify problems or opportunities.
3. Generate and test improvements.
4. Ship the winners.
5. Measure impact.
6. Repeat.

This is not autonomy from engineers. It is autonomy from the user instruction as the starting point. Engineers still set the direction, taste, and guardrails. The loop handles the repetitive execution.

What the Loop Still Does Not Do for You

Loops change the work. They do not remove you from it. Three problems get sharper as the loop gets better, not easier.

Verification Is Still on You

A loop running unattended is also a loop making mistakes unattended. The split between maker and checker helps, but “done” is still a claim, not a proof. Your job is to ship code you confirmed works. The loop can propose. You must approve.

Comprehension Debt Grows Fast

The faster the loop ships code you did not write, the bigger the gap between what exists and what you understand. A smooth loop accelerates this gap unless you read what the loop made. Comprehension debt is the silent cost of agentic development.

Cognitive Surrender Is the Real Danger

The most comfortable posture is the most dangerous. When the loop runs itself, it is tempting to stop having an opinion and accept whatever it gives back. Two people can build the same loop and get opposite results: one uses it to move faster on work they understand deeply; the other uses it to avoid understanding the work at all. The loop does not know the difference. You do.

That is why loop design is harder than prompt engineering, not easier. The leverage point moved. The responsibility did not disappear.

What I Learned About Automation at Google

At Google, I worked on systems where small improvements compounded over time. The teams that moved fastest were the ones that automated the feedback loop: tests ran on every change, metrics surfaced regressions automatically, and alerts pointed engineers to the right place. The loop was not a replacement for engineers. It was a multiplier for the engineers who designed it.

Loop engineering with AI is the same idea, but the agent is now inside the loop. The human designs the goal, the evaluation, and the constraints. The agent executes the iterations. The multiplier is even larger, but the design responsibility is even more important.

Frequently Asked Questions

Are loops just a way to sell more tokens?

No. Loops use more tokens than one-shot prompts, but they also complete tasks that one-shot prompts cannot. The metric that matters is cost per completed task, not cost per token.

Will loops replace engineers?

No. They replace the manual, repetitive parts of engineering work. Direction, taste, architecture, and judgment still come from humans. The engineers who build loops will outperform the ones who do not.

What makes a loop safe?

Clear evaluation, tight scope, and rollback. The agent should only act within a bounded domain. It should be able to undo its changes. And it should stop when the evaluation signal is ambiguous.

When should I not use a loop?

For tasks with no clear evaluation signal. If you cannot define what “done” means, the loop will drift. Use one-shot prompting or human-in-the-loop workflows instead.

How do I start?

Pick a single, bounded task with a clear pass/fail signal. A failing test, a CI check, or a small benchmark is ideal. Build the loop for that one case before generalizing.

What about token costs?

Loops use more tokens than one-shot prompts. Usage patterns vary widely depending on model choice, number of subagents, and cadence. Sonnet is cheaper and often sufficient for planning and verification. Save Opus for the hardest reasoning. Track cost per completed task, not per token.

Which tool should I use?

The tool matters less than the loop design. Codex and Claude Code now share the same primitives: automations, worktrees, skills, MCP connectors, and subagents. Design the loop so it can run in either environment.

References & Further Reading

1. Loop Engineering — Addy Osmani’s deep dive on designing loops across Codex and Claude Code
2. Claude Code — terminal-based AI assistant with /loop, /goal, and subagent support
3. Anthropic Claude Documentation — official docs on agentic capabilities, context management, and skills
4. OpenAI Codex — agentic coding assistant with automations, worktrees, and connectors
5. METR Evaluation of Long-Horizon Models — research on model capability over extended tasks
6. PostHog Blog: Why We’re Bullish on Loops — the original case for loop engineering
7. Stripe Engineering Blog — stories on large-scale automated migrations

What to Read Next

Loop engineering is the natural next step after spec-driven development. The spec defines the goal and constraints. The loop executes until the spec is satisfied. Together, they form the foundation of reliable AI engineering.

• 📖 Related: Spec-Driven Development with AI: From Vibe Coding to Reliable Engineering
• 📬 Newsletter: Get AI engineering posts like this weekly → Subscribe free
• 📥 Resource: The AI Mastery Roadmap for Software Engineers