Executing model-generated code: a two-layer sandbox

Wed, 29 Jul 2026 00:00:00 +0000

Also in 中文. Part 5 of the Engineering LLM Applications series.

The problem: why model-written code must be executed at all

One class of content in automated reporting cannot avoid code execution: experiments where only raw data exists — .mat field-sweep files, csv band structures — while the existing figures are MATLAB .fig files that cannot be embedded in a PDF. The LLM can read the data format but cannot draw: turning data into a figure requires computation plus rendering, and only code does that. So the design is the same as Code Interpreter: the LLM writes a piece of matplotlib code, the backend executes it in a subprocess, out comes a PNG, and compilation embeds it into the report.

On the agent side this is a single tool, make_figure, with exactly two parameters: {name, code}. The agent calls it from its tool loop once it understands the data; on failure it receives the error and may revise the code and retry; on success the report references the figure via \includegraphics{用图/<name>}. The system prompt sets the rule alongside: if there is data but no figure, plot it — never fabricate images.

The cost is equally direct: this executes arbitrary Python generated by a probabilistic component on the backend — a built-in RCE entry point. This article dissects the two-layer sandbox designed for it, and the measured results.

Threat model: the model cannot tell data from instructions

The attack chain is clear: an untrusted data folder → file contents enter the prompt → prompt injection → the LLM writes malicious code → the backend executes it. The root cause is that an LLM processes data and instructions in the same token stream (in-band signaling) — structurally the same defect as SQL injection (data mixed into queries) and buffer overflows (data mixed into control flow). It cannot be fixed at the model layer: alignment training lowers the probability but offers no guarantee. The industry’s consensus path is therefore to assume the injection will succeed and control the damage.

The engineering corollary: the security design must not depend on the assumption that “the model won’t write bad code.” The gates must sit after the code leaves the model and before it has any effect. We built two layers: static interception before execution and runtime isolation — the first blocks known-dangerous patterns before they run, the second assumes the first will leak and minimizes what leaked code can do.

Layer 1: the AST allowlist — static interception before execution

ast_check parses the model’s code into an AST (a parse failure is rejected as a syntax error) and walks every node; any violation rejects the whole submission:

Import allowlist (17 modules): only data/plotting modules — numpy/scipy/matplotlib/pandas/math/csv/json and the like — with both import and from ... import compared by top-level package name. The stance is allowlist, not blocklist: the criterion is “what does a plotting task need,” not “what would an attacker use.”
Dangerous-name blocklist (37 names, checked on bare Name nodes, not just imports): os/sys/subprocess/socket/ctypes/importlib/pickle/pathlib/glob… — any appearance rejects. This rule is not redundant: the execution wrapper pre-imports os to implement file navigation, so model code can call os.remove without ever writing import os. Usage must be intercepted, not merely importing.
Forbidden builtins (17): eval/exec/compile/__import__/open/getattr/globals… — banned both as calls and as bare-name references, so aliasing like e = eval is rejected too.
Dunder escapes: any __xx__ attribute access is rejected, closing the classic ().__class__.__subclasses__() escape chain:

elif isinstance(node, ast.Attribute):
 if node.attr.startswith("__") and node.attr.endswith("__"):
 return False, f"禁止访问 {node.attr}"

Rejection is not a dead end. The error returned to the agent is instructional: it states which libraries are allowed, that data is read via srcpath('filename'), that output is saved via plt.savefig(OUT), and what is banned. The agent takes this deterministic signal, rewrites, and retries — the same closed-loop pattern as compile self-repair: feed the output of a deterministic check back to the model.

Layer 2: runtime isolation — nothing left to damage after a leak

Layer 1 is static analysis over text and can, in principle, miss. Layer 2 assumes it already has, and shrinks the execution environment to the minimum:

Measure	Damage it targets
Environment built from scratch: only the allowlisted variables PATH/MPLBACKEND/MPLCONFIGDIR/HOME/LANG	API keys and proxy settings drop out automatically instead of leaking via the environment
`_GUARD` preamble injected before user code: `os.system` replaced with a lambda raising PermissionError, `sys.modules['subprocess']` set to None, `socket.socket` disabled	Command execution and network egress after a static miss
Isolated cwd: a fresh `mkdtemp` directory per run, deleted afterwards	Writing into host directories
`python -I`: isolated mode	Injection via PYTHONPATH / user site-packages
90-second timeout	Infinite loops and resource exhaustion
Source-data directory mounted read-only by compose (container deployment)	Tampering with source data

Building the environment from scratch has a platform cost: on Windows, a CPython subprocess needs SystemRoot and related system variables (crypto RNG, system DLLs), and matplotlib needs a writable MPLCONFIGDIR/TEMP — these must be explicitly back-filled, or figure generation fails unconditionally in the Windows distribution.

There is also an easily missed companion design: every ban must ship with a replacement. The allowlist bans os/glob/pathlib, so the model cannot list directories or join paths — offering no substitute would force it into violations. The wrapper therefore injects three navigation primitives, all confined to the source-data directory SRC: srcpath('relative-name') for paths, listdir(sub='') for directory listings, and walkfiles() for a recursive list of all relative filenames; the tool description spells out both constraints and usage so the model writes compliant code on the first attempt. On failure, only the last 1500 characters of stderr are returned — error messages are tokens too.

flowchart TD
 A["agent calls make_figure {name, code}"] --> B{"Layer 1: ast_check static allowlist"}
 B -->|reject| E["Instructional error: allowed and banned items spelled out"]
 E --> F["agent rewrites code, retries"]
 F --> B
 B -->|pass| W["Wrapper: pre-import np/plt/loadmat, inject SRC/OUT/srcpath/listdir/walkfiles + _GUARD"]
 W --> R["Layer 2: isolated subprocess<br/>from-scratch env · isolated cwd · python -I · 90s timeout"]
 R -->|"OUT exists and non-empty"| P["PNG into session figure dir, embedded at compile time"]
 R -->|failure or timeout| E

Measured: seven classes of malicious samples, all blocked before execution

Verified by real runs inside the container. Two legitimate paths passed: basic matplotlib plotting, and plotting real data read via srcpath. Seven classes of malicious samples were all rejected by the AST layer before execution:

Malicious sample	Rule hit
`os.system` command execution	Dangerous-name blocklist (os is pre-imported; bare names are intercepted, not imports)
`os.remove` deleting files (no import — uses the pre-imported os)	Same as above
`import socket` for egress	Import allowlist
`eval` dynamic execution	Forbidden builtins
`__import__` bypassing the import statement	Forbidden builtins
Dunder escape (`__class__`/`__subclasses__`)	Dunder attribute interception
`open` reading /etc/passwd	Forbidden builtins

Layer 2 was never reached by this sample set — which is exactly as intended: a backstop’s value lies in unknown samples, not in the test set.

Two unexpected payoffs: the security constraint pays back at packaging time

For standalone distribution the backend is frozen into a binary with PyInstaller. Two interactions with the sandbox could have been deep pits; the allowlist made both shallow.

Payoff 1: a finite, known import set makes collect_all a precise fix. PyInstaller collects dependencies by static analysis, but the plotting code inside the sandbox is a string that exists only at runtime — whatever it imports is invisible to the packager. With an unbounded import set the problem is unsolvable; the AST allowlist, however, pins it down to a finite known set, so running collect_all over numpy/scipy/matplotlib and the rest captures everything. Measured on Windows: the frozen onedir output is ~260MB (30MB exe), with selftest reporting imports 10/10 and savefig 18250 bytes — the scientific stack is fully usable inside the frozen bundle.

Payoff 2: self-dispatch around the frozen sys.executable. In development the execution command is [sys.executable, "-I", script]; after freezing, sys.executable is this binary rather than python, and exe -I script would be interpreted as a server launch and fail. The fix: when run_plot detects is_compiled(), it instead sets the environment variable PDFAGENT_PYRUN=<script path> and launches another copy of the same exe; the entry point run_server._pyrun() sees the variable, executes the script via runpy.run_path, and exits without starting the server. Process isolation, the scrubbed environment, the timeout, the AST allowlist, and _GUARD all remain — only the -I flag is lost. The dispatch is pinned by the two cases in test_pyrun.py (with the variable set, the script runs; without it, no-op).

Where not to do this

This is not a full sandbox. There is no microVM and no container-level isolation; the subprocess shares the host’s kernel and filesystem permissions — a combined attack that bypasses the AST layer and evades _GUARD exists in theory. The project states its position in the module docstring: it targets the local single-user scenario (the code plots the user’s own local data — equivalent to the user running a script by hand), where two layers suffice; for untrusted folders or multi-user service, the real fix is one-shot, network-less container execution.
The allowlist sacrifices expressiveness. getattr and open are banned; h5py and similar libraries are not on the list, so HDF5-class data requires explicitly extending the allowlist — and re-assessing the new library’s capability surface (can it make network requests, can it write files). Every notch the allowlist widens, the weaker Layer 1’s guarantee becomes.
Domain restriction is the precondition. The approach works because the task domain is narrow: “data plotting” needs a small, stable module set. For general-purpose code execution — an agent freely writing arbitrary tool scripts — the allowlist would widen until meaningless; go straight to container/microVM isolation, with the AST layer at most as a heuristic pre-check.
Layer 2 couples to platform details. Building the environment from scratch means discovering every implicit platform dependency yourself: Windows system variables, matplotlib’s writable config directory. Projects distributing cross-platform should budget for verification by real runs, not by reasoning.

Security | kaguc — Writing to understand systems.