The idea in one paragraph.
Treat code as the action space and execution as the observation: a standardised, sandboxed environment where interactive coding agents can be benchmarked with real execution feedback.
THE MACHINERY
Code as action space
The agent emits executable code rather than choosing from a fixed action list.
Execution as observation
stdout, stderr, and exit codes are the observation the next step conditions on.
Containerised task environments
Each task runs in a reproducible sandbox with a programmatic reward.
What the paper reports.
Findings are reproduced as concise corpus summaries, with their original confidence labels. They are not independent replications.
Frames interactive coding as a standard reinforcement learning environment, with code as actions and execution feedback as observations.
Inspect a source excerpt
a lightweight, flexible, and easy-to-use framework of interactive coding as a standard reinforcement learning (RL) environment, with code as actions…Read the passage in context on arXiv ↗
Uses self-contained Docker environments for safe and reproducible execution, with Bash, SQL, and Python as action spaces.
Extensible enough to create new task types such as Capture the Flag, which is inherently multi-step and multi-language.
LIMITATIONS / CORPUS READING
Where to keep your judgement.
- Rewards are task-specific and hand-written; they do not generalise to open-ended work.
- Sandboxing constrains the very open-endedness that makes code-as-action powerful.
Why it belongs in this atlas.
The rung most people skip. Once actions are code, the tool list stops being a fixed vocabulary - the agent can compose new capability at run time. The talk is candid about the resulting ambiguity: is an emitted function a tool or a skill?