# Wiring external signals into the loop: compile-and-repair

> Run generated LaTeX through a real compiler and feed the errors back to the model — on tasks where errors are hard to self-detect, an external verification signal is something self-reflection cannot replace. This article dissects the loop's five engineering constraints: a two-round cap, error-fingerprint no-progress detection, a wall-clock budget, a ~8k-token minimal repair context, per-session compile directories — and when not to build this loop at all.

- Canonical (HTML): https://kaguc.com/blog/compile-self-repair/
- Date: 2026-07-29


*Also in [中文](/blog/compile-self-repair-zh/). Part 6 of the Engineering LLM Applications series.*

## The problem: models cannot fix errors they cannot see

Scenario: an autonomous agent generates a LaTeX lab report, and xelatex fails. Who fixes it?

The reflex is to ask the model to "check it again." The evidence against that path is specific. One of Self-Refine's conclusions (arXiv:2303.17651) is that **on tasks where errors are hard to self-detect, pure self-reflection is nearly useless, and gains only return once an external verification signal is wired in**. The systematic review in *When Can LLMs Actually Correct Their Own Mistakes* (TACL 2024) is blunter: prompting-only self-critique does not improve — and can degrade — arithmetic and code tasks, and the flattering self-correction numbers in the literature mostly leaned on an oracle (stopping only when the known-correct answer appears). Compile errors are the canonical hard-to-self-detect case: the model rereads the `\def` line it just wrote and cannot see that a `%` inside the braces comments out the closing brace — but xelatex sees it, and its error report cannot be faked.

So the right question is not "how do we make the model self-check" but "how do we wire an objective external verifier into the generation loop." This article dissects the compile-and-repair loop we shipped in an AI writing agent — **really compile after generation; on failure, extract the errors and feed them back for a repair round** — and the engineering constraints that keep this deceptively simple loop under control in production.

## From patching to really compiling

Before the compiler was wired in, the system already had a deterministic sanitizer layer ([part 3 of this series](/blog/output-sanitizing/)): `\mathbf`→`\symbf`, `%` comments inside `\def` lines, backfilling template variables the model forgot to set — the known pitfalls that rules can encode. But patches cannot enumerate the long tail. The devlog states the goal of this step plainly: "not patching with the sanitizer, but having the agent really compile after generating the report, feed the xelatex errors back for a round of its own fixes on failure, and push the first-pass success rate to the max."

That fixed the division of labor: **rules handle the known pitfalls (zero tokens); the compiler backstops the long tail (one LLM call per round)**. In practice the sanitizer has already cured the common traps, so the first compile check usually passes outright; the repair branch only catches what remains. The loop itself is three steps:

1. `compile_tex` — assemble a complete working directory (article.tex + cls + bib + figures), run xelatex → bibtex → xelatex ×2, return `{ok, pdf_path, log}`;
2. `extract_errors` — pull from the log the lines starting with `!`, the `l.` line-number lines, and Runaway / Emergency stop lines, capped at 1800 characters — never feed back the whole log;
3. on failure, feed the error excerpt back to the model, demand the full revised source, at most 2 rounds.

The cap of 2 is not arbitrary: a survey finding (*How Many Tries*, arXiv:2604.10508) across 7 models is that the first two rounds capture 76–95% of the achievable gain, with near-zero return from round three on; Self-Refine likewise plateaus by round 3. Extra rounds only multiply wall-clock and token spend. One further detail: the second round's repair output is not re-verified inside the loop — the repaired draft is kept, and final verification is left to the user-side manual compile/export path.

A counter-intuitive design point: the repair is a **full-document rewrite**, not a scoped patch of the kind used in part 7. The reason is a property of LaTeX errors — they are often decoupled from the root cause, and line numbers are unreliable (*LaTeX Compilation Challenges*, arXiv:2603.02873), so pinpoint fixes at the reported line easily patch the wrong place. Scoped editing presupposes deterministic targeting, and compile errors are precisely where that presupposition fails.

## Termination: fingerprint, round cap, wall clock

The biggest production risk of a self-repair loop is not failing to fix — it is **failing to stop**. The consensus on loop governance is that "when to stop" must not be left to the model's discretion; an outer controller enforces it with deterministic criteria. There are three gates:

**The no-progress gate.** After each failed round, fingerprint the errors — the first 120 characters of the first `!` error line (LaTeX's anchor line); if it equals the previous round's fingerprint, declare no progress, stop immediately, keep the current draft:

```python
def _error_fingerprint(errs: str) -> str:
    """Compile-error fingerprint: first '! …' error line (LaTeX's anchor),
    else the first 120 chars. Used for no-progress detection."""
    for line in (errs or "").splitlines():
        s = line.strip()
        if s.startswith("!"):
            return s[:120]
    return (errs or "").strip()[:120]
```

If the same error survived a repair round, this model has no answer for this error; feeding it the same report again will most likely produce the same failure, and burning more tokens has no expected return.

**The round cap.** `for attempt in range(2)` — a hard limit.

**The wall-clock gate.** A single compile runs up to 4 subprocess passes at 180 seconds each — a token budget cannot contain CPU time. `COMPILE_WALL_BUDGET` (default 600 seconds) is checked before each round; on expiry, stop and keep the current draft. The config comment states the motive: prevent multi-round self-repair from hogging the customer machine's CPU.

All three gates are deterministic criteria executed on the Python side; the model has no say in whether the loop stops — the same stance as the tool-loop circuit breaker in [part 4](/blog/agent-loop/).

## Minimal repair context: ~8k tokens, not 40–60k

How the repair call's context is constructed is the single most money-saving decision in this loop. The intuitive approach is to reuse the agent's full exploration history — surely the model "fixes better knowing more"? The opposite holds. The judgment recorded in the code comment: **95% of compile errors are template-level syntax problems** (balancing, command misuse, template variables), and fixing them requires exactly three things:

- the full current draft;
- the error excerpt (≤1800 characters);
- the formatting hard rules (no `%` comments inside `\def` braces; bold vectors with `\symbf`; no bold inside accents; balance braces/environments/`$`; end with a top-level `\end{document}`).

About 8k tokens in total. Reusing the full exploration messages would carry 40–60k tokens of dead weight — source material, tool-call records, multiple draft versions — none of it contributing to fixing one brace; switching to a fresh minimal context saves 30–50k tokens per repair round. The system prompt narrows the mandate to a single sentence:

> You are a LaTeX compile-repair assistant: fix compile errors only — do not rewrite content, do not add or remove sections, do not touch data.

This is not only about cost: the measurements in *Revisit Self-Debugging* (arXiv:2501.12793) show label-only feedback beating verbose detailed feedback — inaccurate long context introduces ambiguity and lowers scores. Commercial products are isomorphic: Overleaf's commercial Error Assist also deliberately minimizes the repair context, sending only the full error, the relevant code lines, and a file-name list (product-page information, not independently verified).

## Concurrency isolation: a compile directory per session

A plain constraint you only learn by tripping on it: the compile working directory must be isolated per session. `compile_tex` writes article.tex into the directory, copies the cls and figures, and rmtree-rebuilds the figure subdirectory during assembly — two concurrent generation runs sharing one directory will overwrite each other's source, delete each other's figures, and pollute each other's logs. After partitioning the repair directory as `_repair/<session_id>`, each run writes its own. The lesson generalizes: **a verifier is a side-effecting process, not a pure function** — when wiring an external verifier into a loop, isolation is a default, not an optimization.

## The whole picture, quantified

```mermaid
flowchart TD
    A[Final draft] --> W{Wall-clock budget exceeded?}
    W -->|yes| S[Stop, keep current draft]
    W -->|no| C[compile_tex: real compile]
    C -->|ok| P[PDF produced]
    C -->|fail| E[extract_errors: pull ! error lines]
    E --> F{Fingerprint same as last round?}
    F -->|same = no progress| S
    F -->|different| R[Minimal-context repair call ≈8k tokens]
    R --> N[2 rounds used?]
    N -->|yes| S
    N -->|no| W
```

| Constraint | Failure mode without it | Implementation and numbers |
|---|---|---|
| Round cap | Grinding on for rounds with near-zero return from round 3 | K=2; first two rounds capture 76–95% of achievable gain (survey figure) |
| No-progress gate | The same error fed back again and again, spinning in place | Fingerprint = first 120 chars of the first `!` line; stop on consecutive match |
| Wall-clock gate | 4 passes × 180s per compile; a token cap cannot contain CPU | `COMPILE_WALL_BUDGET`, default 600s; on expiry keep the current draft |
| Minimal context | 40–60k tokens of exploration-history dead weight per call | Current draft + error excerpt + hard rules ≈8k; saves 30–50k tokens per round |
| Session isolation | Concurrent runs clobber each other's source/figures/logs | Dedicated `_repair/<session_id>` working directory |

## Where not to build this loop

1. **It presupposes a cheap, objective, unfakeable verifier.** Compilers, type checkers, and unit tests qualify; "is the writing good" does not. The stop condition must anchor on an objective signal like "compilation succeeded" — never on the model's self-assessment. Quality dimensions without a verifier belong to evaluation and judges (parts 8 and 9), not to this loop.
2. **Deterministic repair comes first.** Pitfalls that can be written as rules (known syntax traps, missing-figure placeholders, variable backfill) should be cured by the zero-token sanitizer before any LLM repair; the repair loop only backstops the long tail that rules cannot enumerate. Reversing the order means using a probabilistic component for deterministic work: paying a call every time while adding new uncertainty.
3. **Weak models need a non-LLM exit.** Survey data puts weak models' repair rate on deadlocked errors at only about 30% (not independently verified). The loop must ship with a fallback that does not depend on the model — here, export_zip produces a compilable package to hand to Overleaf or a human. Without an exit, the loop degenerates on weak models into "burn the budget, then give up."
4. **When the verifier is expensive, the budget gate precedes the round cap.** Here a single compile is wall-clock-bounded at 4 passes × 180 seconds (at most 720s), so K=2 is affordable; if the verifier is a ten-minute integration test suite, the same K=2 may be unacceptable — the wall-clock budget must become the first gate, including the option of "give up without repairing at all."

## References

- *Self-Refine: Iterative Refinement with Self-Feedback* (arXiv:2303.17651) — self-reflection is nearly useless on tasks where errors are hard to self-detect; gains return with external signals; benefit concentrates in the first 1–2 rounds (its "+20% average" figure not independently verified).
- *When Can LLMs Actually Correct Their Own Mistakes* (TACL 2024) — prompting-only self-critique does not improve, and can degrade, arithmetic/code tasks; flattering self-correction results mostly relied on an oracle.
- *How Many Tries* (arXiv:2604.10508) — the first two rounds capture 76–95% of achievable gain; near zero from round 3.
- *LaTeX Compilation Challenges* (arXiv:2603.02873) — LaTeX errors are often decoupled from root causes; line numbers are unreliable.
- *Revisit Self-Debugging* (arXiv:2501.12793) — label-only feedback beats verbose feedback.
- Overleaf Error Assist — the commercial isomorph of minimal repair context (product-page information, not independently verified).