Voyager (2023) solved agentic tasks with code execution

Two and a half years since the launch of the GPT-3 API, code assistants have emerged as potentially the premier use case of LLMs. The rapid adoption of AI-powered IDEs and prototype builders isn’t surprising — code is structured, deterministic, and rich with patterns, making it an ideal domain for machine learning. Developers actively working with tools like Cursor (myself included) have an exhiliarating yet uncertain sense that the field of software engineering is approaching an inflection point.

Yet there’s a striking gap between understanding and action for today’s code assistants. When provided proper context, frontier LLMs can analyze massive enterprise codebases and propose practical paths towards sophisticated, large-scale improvements. But implementing changes that impact more than a small set of files with modern AI assistants is fundamentally infeasible. The good news is that for focused, file-level changes, we’ve found real success: AI-powered IDEs (Windsurf, Cursor) are transforming how developers write and review code, while chat-based assistants are revolutionizing how we bootstrap and prototype new applications (via tools like v0, lovable.dev, and bolt.new).

However, there’s a whole class of critical engineering tasks that remain out of reach - tasks that are fundamentally programmatic and deal with codebase structure at scale. A significant amount of effort on modern engineering teams is directed towards eliminating tech debt, managing large-scale migrations, analyzing dependency graphs, enforcing type coverage across the codebase, and similar tasks that require a global view of a codebase. Today’s AI assistants can fully understand these challenges and even propose solutions, but they lack the mechanisms to actually implement them. The intelligence is there, but it’s trapped in your IDE’s text completion window.

The bottleneck isn’t intelligence — it’s tooling. The solution requires letting AI systems programmatically interact with codebases and software systems through code execution environments. Code execution environments represent the most expressive tool we could offer an agent—enabling composition, abstraction, and systematic manipulation of complex systems. By combining code execution environments with custom APIs that correspond to powerful large-scale operations, we can unlock a new set of tasks in which agents can be significant contributors. When paired with ever-improving foundation models, this will lead to a step function improvement for code assistants, enabling their application in an entirely new set of valuable tasks.

​
Beating Minecraft with Code Execution

In mid-2023, a research project called Voyager made waves: it effectively solved Minecraft, performing several multiples better than the prior SOTA. This was a massive breakthrough as previous reinforcement learning systems had struggled for years with even basic Minecraft tasks.

While the AI community was focused on scaling intelligence, Voyager demonstrated something more fundamental: the right tools can unlock entirely new tiers of capability. The same GPT-4 model that struggled with Minecraft using standard agentic frameworks (like ReAct) achieved remarkable results when allowed to write and execute code. This wasn’t about raw intelligence—it was about giving the agent a more expressive way to act.

The breakthrough came from a simple yet powerful insight: let the AI write code. Instead of limiting the agent to primitive “tools,” Voyager allowed GPT-4 to write and execute JS programs that controlled Minecraft actions through a clean API.

// Example "action program" from Voyager, 2023
//   written by gpt-4
async function chopSpruceLogs(bot) {
  const spruceLogCount = bot.inventory.count(mcData.itemsByName.spruce_log.id);
  const logsToMine = 3 - spruceLogCount;
  if (logsToMine > 0) {
    bot.chat("Chopping down spruce logs...");
    await mineBlock(bot, "spruce_log", logsToMine);
    bot.chat("Chopped down 3 spruce logs.");
  } else {
    bot.chat("Already have 3 spruce logs in inventory.");
  }
}

This approach transformed the agent’s capabilities. Rather than being constrained to atomic actions like equipItem(...) (this would be typical of “traditional” agent algorithms, such as ReAct), it could create higher-level operations like craftShieldWithFurnace() through composing the atomic APIs. Furthermore, Wang et al. implemented a memory mechanism, in which these successful “action programs” could later be recalled, copied, and built upon, effectively enabling the agent to accumulate experience.

As the Voyager authors noted:

“We opt to use code as the action space instead of low-level motor commands because programs can naturally represent temporally extended and compositional actions, which are essential for many long-horizon tasks in Minecraft.”

​
Code is an Ideal Action Space

What these authors demonstrated is a fundamental insight that extends far beyond gaming. Letting AI act through code rather than atomic commands will lead to a step change in the capabilities of AI systems. Nowhere is this more apparent than in software engineering, where agents already understand complex transformations but lack the tools to execute them effectively.

Today’s productionized code assistants operate though an interface where they can directly read/write to text files and perform other bespoke activities, like searching through file embeddings or running terminal commands.

In the act via code paradigm, all of these actions are expressed through writing and executing code, like the below:

# Implement `grep` via for loops and if statements
for function in codebase.functions:
   if 'Page' in function.name:  
      
      # Implement systematic actions, like moving things around, through an API
      function.move_to_file('/pages/' + function.name + '.tsx')

Provided a sufficiently comprehensive set of APIs, this paradigm has many clear advantages:

  • API-Driven Extensibility: Any operation that can be expressed through an API becomes accessible to the agent. This means the scope of tasks an agent can handle grows with our ability to create clean APIs for complex operations.

  • Programmatic Efficiency: Many agent tasks involve systematic operations across large codebases. Expressing these as programs rather than individual commands dramatically reduces computational overhead and allows for batch operations.

  • Composability: Agents can build their own tools by combining simpler operations. This aligns perfectly with LLMs’ demonstrated ability to compose and interpolate between examples to create novel solutions.

  • Constrained Action Space: Well-designed APIs act as guardrails, making invalid operations impossible to express. The type system becomes a powerful tool for preventing entire classes of errors before they happen.

  • Objective Feedback: Code execution provides immediate, unambiguous feedback through stack traces and error messages—not just confidence scores. This concrete error signal is invaluable for learning.

  • Natural Collaboration: Programs are a shared language between humans and agents. Code explicitly encodes reasoning in a reviewable format, making actions transparent, debuggable, and easily re-runnable.

​
Code Manipulation Programs

For software engineering, we believe the path forward is clear: agents need a framework that matches how developers think about and manipulate code. While decades of static analysis work gives us a strong foundation, traditional code modification frameworks weren’t designed with AI-human collaboration in mind - they expose low-level APIs that don’t match how developers (or AI systems) think about code changes.

We’re building a framework with high-level APIs that correspond to how engineers actually think about code modifications. The APIs are clean and intuitive, following clear principles that eliminate sharp edges and handle edge cases automatically. Most importantly, the framework encodes rich structural understanding of code. Consider this example:

# Access to high-level semantic operations
for component in codebase.jsx_components:
    # Rich structural analysis built-in
    if len(component.usages) == 0:
        # Systematic operations across the codebase
        component.rename(component.name + 'Page')

This isn’t just string manipulation - the framework understands React component relationships, tracks usage patterns, and can perform complex refactors while maintaining correctness. By keeping the codebase representation in memory, we can provide lightning-fast operations for both analysis and systematic edits.

The documentation for such a framework isn’t just API reference - it’s education for advanced intelligence about how to successfully manipulate code at scale. We’re building for a future where AI systems are significant contributors to codebases, and they need to understand not just the “how” but the “why” behind code manipulation patterns.

Crucially, we believe these APIs will extend beyond the codebase itself into the broader software engineering ecosystem. When agents can seamlessly interact with tools like Datadog, AWS, and other development platforms through the same clean interfaces, we’ll take a major step toward autonomous software engineering. The highest leverage move isn’t just giving agents the ability to modify code - it’s giving them programmatic access to the entire software development lifecycle.

​
Codegen is now OSS

We’re excited to release Codegen as open source Apache 2.0 and build out this vision with the broader developer community. Get started with Codegen today or please join us in our Slack community if you have feedback or questions about a use case!

Jay Hack, Founder

Was this page helpful?

All Posts