From chat to agent: tool loops and collapse breakers

Also in 中文. Part 4 of the Engineering LLM Applications series. The series is grounded in the implementation and measurements of a production-grade AI writing agent (FastAPI + React + Tauri, 351 test cases).

The problem: handing the loop to the model

The first three articles of this series were about single-call engineering: controlling input, assembling prompts, sanitizing output. The watershed of an agent is that what to do next is also handed to the model: give it a tool set, execute its calls, feed the results back into the message list, and call again — until the task is done.

The concrete scenario for this article: give the agent a data folder whose file names and contents are unknown in advance, and have it read files on its own, figure out what experiment this is, and write the report. The naive implementation is a while True tool loop, and in production it fails in three ways:

1. Untrusted paths. The model decides which file to read; a single ../ in an argument reads outside the data directory.
2. No natural endpoint. “How much exploration is enough” is the model’s judgment call; there is no upper bound on iterations, and therefore none on cost.
3. Loop collapse. The most insidious class: in long contexts, the model degenerates into calling the same tool with identical arguments. A public case, qwen-code issue #4695, records 43 consecutive git status calls consuming 8.9M tokens in one session; the deepseek-v4-pro tier we use is also prone to collapsing into this pattern in long contexts. SDK retries cannot help — retries only cover network errors and rate limits, while each of these calls “succeeds” at the API layer. The breaker has to live on the client side.

The rest of this article walks through five mechanisms in build order. The implementation uses standard OpenAI-compatible tool calling (DeepSeek by default; any compatible endpoint can be swapped in); the entry point run_agent is a generator that yields events one by one for SSE streaming.

Mechanism 1: a minimal tool set with a path guard

The exploration phase has five tools: list_dir, read_file, read_image (a local vision model extracts readings from screenshots and figures), make_figure (executes model-written matplotlib code to produce plots), and finish_exploration. Three design points:

Segmented continuation instead of stuffing everything in at once. read_file clips text at READ_CHARS = 7000 characters and, when it clips, appends one line: “truncated — use offset/limit to continue reading by lines”. Truncation is not an error; it is a decision point for the model: there is more, and whether it is worth reading further is the model’s trade-off to make. It also imposes a constraint on the loop-breaker fingerprint later — continuation reads with different offsets are legitimate behavior and must not be flagged.

Explicit phase transition. finish_exploration turns “exploration is done” from an implicit signal (the model simply stops calling tools) into an explicit call: the schema requires experiment / goal / key_findings, with three optional slots for assignment requirements, methodology notes, and a reference report. The phase transition leaves a structured record — and gives the breaker a well-defined exit to steer toward.

Every path entry goes through _safe.

def _safe(rel, root=None):
    root = root or base_docs_root()
    if not root:
        return None, None
    t = os.path.normpath(os.path.join(root, rel or ""))
    if not (t == root or t.startswith(root + os.sep)):
        return None, None
    return root, t

After normpath joining, the resolved path must still be inside the root; an escape returns (None, None), and the tool layer turns that into a “path does not exist or is out of bounds” string returned as the tool result — the model sees it and can correct itself, and the main flow never raises. The make_figure tool description also spells out hard sandbox rules (no importing os/sys/subprocess and the like, no open()/eval/exec, no double-underscore attributes); its two-layer sandbox implementation is the subject of the next article.

Mechanism 2: a zero-LLM scan before exploration starts

Before the model begins exploring, scan_folder performs a deterministic recursive scan: files are coarsely bucketed by extension into documents / images / data / other, returning the bucketed listing and counts — zero LLM calls, zero tokens. It serves two purposes: an instant whole-folder snapshot for the user (the detail of the first step event), and deterministic input for the later material-gap check. The principle is the same as in Part 11: “what is in the folder” is a question code can answer, and the model’s exploration budget should be spent on “which files to read and what to make of them”. One cost note in passing: the exploration tool loop runs on the cheap model tier (reading files and deciding what to read is simple work); drafting and reflection use the main model.

Mechanism 3: two lines of defense — a budget cap and a collapse breaker

MAX_TOOL_ITERS = 20 is a budget cap, not a defense: a collapsed agent will burn all 20 iterations on repeated calls and produce nothing. The real defense is _LoopBreaker, built around a call fingerprint:

def _tool_fingerprint(name, args):
    """Tool-call fingerprint = name + full arguments (sorted JSON)."""
    try:
        return name + "|" + json.dumps(args, ensure_ascii=False, sort_keys=True)
    except Exception:
        return name + "|" + str(args)

Three design constraints: full arguments — offset=0 and offset=100 are two different fingerprints, so legitimate segmented reads are never flagged (a unit test pins this property); sorted serialization — argument key order does not affect the verdict; never raises — serialization failure falls back to str(). Wrap-up tools like finish_exploration are excluded from counting.

The thresholds come in two tiers. Third occurrence of the same fingerprint (_LOOP_WARN): inject one wrap-up prompt — “You are repeatedly calling the same tool with the same arguments. What you have read is sufficient; call finish_exploration immediately and state your conclusions” — with nudge_once guaranteeing a single injection per run, because the injection itself occupies context and repeated injections would accelerate the collapse. Fifth occurrence (_LOOP_STOP): hard-break out of the exploration loop. The break is not a crash: the agent yields a “repeated identical tool call detected — exploration cut off, writing from what has been read” event and proceeds to the drafting phase with the material accumulated so far. A degraded deliverable plus a human reviewer beats zero output after the budget is exhausted.

The whole loop still sits inside for _ in range(MAX_TOOL_ITERS) — the two defenses hold independently. The breaker has an end-to-end test: a mock client that, whenever tools are passed, always returns the same read_file(a.txt) call (simulating total collapse); the test asserts that the break event must appear and that the number of create calls is ≤ 6 — far below the cap of 20. The threshold sequence (first two ok, third nudge, fifth stop) and the fingerprint properties are pinned by separate unit tests.

Mechanism 4: the inverse failure in drafting — tool markup without tools

After exploration, drafting begins, and this phase passes no tools. But the model may still emit its internal tool-call markup (DeepSeek’s DSML) as body text — the generation “succeeds” on the surface, and the resulting report is garbage. _complete_report validates output with three deterministic criteria: contains \documentclass, contains no DSML, contains no tool_calls. On failure it injects a reinforcement instruction (“You have no tools available right now… output the complete LaTeX source directly”) and retries, up to 3 times. This is the output-sanitizing approach of Part 3 extended into the agent context: a structurally invalid output is not patched textually — the constraints are tightened and the call is retried.

Mechanism 5: deterministic progress — pre-enumerated milestones and explicit stages

A streaming agent’s progress bar cannot run on guesswork. This pipeline is “process as configuration”: scan → explore → [figures] → [gap] → draft → reflect×N → bib → [compile], with the optional steps switched by the template. So _plan_pipeline enumerates the milestone sequence before the run starts, from configuration alone; M = len(plan), every milestone event carries index/total, and the progress bar has a fixed denominator from the first second. Two consistency details:

• The gates for optional steps must match the runtime decisions verbatim: has_xelatex is probed exactly once, and the plan and the runtime compile gate reuse the same result, so the two can never diverge;
• the stage name is passed explicitly rather than looked up from _plan[_pi-1]: the reflection loop may break early on convergence, and a skipped milestone would make the positional lookup mislabel the subsequent bib / compile milestones as reflect.

One known small deviation is documented honestly in the code comments: the gap step also depends at runtime on whether material exists and may actually be skipped — the plan counts it anyway, and the completion event backfills to 100%.

Measurements

Real-run validation (devlog record, DeepSeek v4-pro, an electromagnetics field-scan dataset): the agent ran list_dir automatically and made multiple rounds of read_file (a PDF, band.csv, scan parameter files, and more), and its autonomous identification of the experiment was correct; the draft was 16,773 characters, and after one reflection-revision round the final report was 20,742 characters across 9 sections, with the full run taking 745 seconds. That validation ran on the initial version whose tool set was only list_dir / read_file / finish_exploration; read_image and make_figure were added later.

Where not to do this

1. An exact fingerprint catches total collapse, not wandering. Low-value calls to the same tool with varying arguments (reading irrelevant files one after another) never trip the breaker; that class is bounded by the budget cap and prompt quality. Fuzzing the fingerprint (ignoring some arguments) would widen coverage but misfire on legitimate continuation reads — we chose narrow and precise.
2. The nudge works only while the model still follows instructions. The wrap-up prompt helps with mild circling; once the context has collapsed severely, the model no longer responds to injections and only the hard break matters — which is exactly the rationale for the two-tier thresholds.
3. A degraded deliverable requires a human in the loop. A report written from partial material is only worth something if someone reviews it downstream. If the agent’s output is executed automatically and partial information can cause harm, the breaker policy should be fail-the-whole-run rather than degrade.
4. Pre-enumerated progress requires a fixed pipeline. _plan_pipeline works because the pipeline structure is settled before the run starts; an open-ended agent whose plan is generated dynamically by the model has no a-priori milestone sequence, and progress degrades to event counting.
5. The thresholds are not universal constants. 20 / 3 / 5 were tuned to the scale of “read a folder, write a report” and have not been validated on other task shapes.

References

• qwen-code issue #4695 — a public case of tool-loop collapse (43 repeated git status calls / 8.9M tokens in one session).