跳到正文
原文
Microsoft Research· Aaron J. Li, Weijia Xu, Chandan Singh, Jeevana Priya Inala, Bin Yu, Ion Stoica, Michel Galley, Jianfeng Gao·· 16 小时前AI 评分57

微软研究院开源 TokenEconomizer 插件,AI 编码智能体成本平均降 18.4–30.9%

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

文章要点
  • 1微软研究院等机构开源 TokenEconomizer 插件,让编码 Agent 复用仓库历史经验,降低后续任务 token 成本
  • 2官方评测 277 个 SWE-Bench 修 bug 任务,五种 Agent 与模型组合平均省 18.4–30.9%,成功率基本不变
  • 3对 Claude Code 的剖析显示三类浪费,同仓库 30% 工具调用重复旧操作,23 个工具中 16 个从未被调用
  • 4插件靠微技能笔记、工具裁剪、失败循环干预三个机制省钱,仓库累计成本到第 6 个任务起低于默认基线
决策要点

编码 Agent 的成本优化有了经验复用这条新路径,在同一代码库反复作业的场景下,省 token 不必以成功率为代价。

  • 影响:对长期在同一代码库使用编码 Agent 的研发团队,token 开销有两到三成压降空间;经验需逐会话积累,前几个任务成本可能高于默认,到第 6 个任务才见节省;插件不改 Agent 与模型,试错成本主要是评测投入。
  • 参考:可考虑先统计现有编码 Agent 的 token 账单与成功率做基线,再在任务量稳定的固定仓库上试点这款开源插件;值得把成功率设为红线指标,官方对照显示压低模型推理力度省钱相近或更多但性能明显下降。
  • 依据:18.4–30.9% 等数字来自论文作者在 277 个 SWE-Bench 任务、五种 Agent 与模型组合上的自测,属官方口径暂无独立验证;已计入插件自身模型调用成本。
正文
Six coding sessions with shrinking cost bars send their logs to TokenEconomizer, which uses micro-skills, tool pruning and failure-loop intervention to guide an unchanged coding agent through the next session.

At a glance

  • Developers use AI coding agents in the same codebases day after day, yet agents often start each session from scratch: rediscovering familiar facts, loading unused tools, and repeating unsuccessful approaches.
  • TokenEconomizer is an open-source plugin that learns from an agent’s past sessions in a repository to reduce waste in future sessions, without modifying the agent or its underlying model.
  • Across 277 real bug-fixing tasks, TokenEconomizer reduced costs by 18.4–30.9% across multiple AI coding agents. Costs fell in every repository tested, while task success rates remained broadly stable.

AI coding agents are increasingly helping developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same repositories, giving agents an opportunity to learn from previous work. Without that learning, an agent may repeatedly rediscover how to run tests, load tools it never uses, or spend dozens of turns revisiting the same error. 

In our paper, TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents, we ask a simple question: can experience in a repository help an agent solve future tasks more efficiently? TokenEconomizer, an open-source plugin from UC Berkeley, Microsoft Research, and Microsoft Security, turns completed sessions into guidance for subsequent ones. It reduces waste without changing the agent or its underlying model, so developers can use it alongside the coding agents they already rely on.

Each finished session teaches TokenEconomizer something about the repository, so later sessions cost less tokens.
Figure 1. Each finished session teaches TokenEconomizer something about the repository, so later sessions cost less tokens. 

Where the tokens go

We first profiled Claude Code on 60 tasks from SWE-Bench, a benchmark of real bug fixes in open-source Python projects, running each repository’s tasks in sequence. Three kinds of waste recurred:

  • Rediscovering the same facts. After a repository had accumulated at least five completed tasks, 30% of tool calls repeated an action from an earlier task in that repository. Only 7% matched an action in another repository’s history.
  • Paying for unused tools. The agent received 23 tool descriptions on every turn, but 16 were never called.
  • Getting stuck on failures. The 17% of tasks that failed accounted for 32% of the spending.

We also tested three popular open-source plugins, each built around a fixed cost-saving strategy. Each lowered costs in some repositories but not others. Together, these findings suggest that effective cost control must adapt to the repository, rather than apply the same strategy everywhere.

How TokenEconomizer works

After each session, TokenEconomizer reads the session’s log and updates what it knows about the repository. It then acts through three mechanisms, using only interfaces the agent already provides, as illustrated in Figure 2.

Micro-skills capture repository-specific knowledge in one-line notes. TokenEconomizer identifies recurring patterns across sessions and proposes corresponding notes. They are added to the agent’s starting context only after several sessions have confirmed the pattern, and retires the note if later evidence contradicts it.

Tool pruning removes tools that repository history suggests the agent is unlikely to use, while always retaining core tools for running commands and editing files. Removed tools can be restored if needed.

Failure-loop intervention watches for an agent that keeps hitting the same failure. A judge model decides whether the agent is truly stuck or still making progress. If it is stuck, the agent gets a short note suggesting a different approach. In the open-source plugin, if the agent is still stuck after the note, TokenEconomizer alerts the developer rather than stopping the work itself.

TokenEconomizer works alongside an unchanged coding agent. Micro-skills add learned facts, tool pruning trims the loaded tools, and failure-loop intervention redirects stuck work. Each finished session updates all three.
Figure 2. TokenEconomizer works alongside an unchanged coding agent. Micro-skills add learned facts, tool pruning trims the loaded tools, and failure-loop intervention redirects stuck work. Each finished session updates all three. 

The controller is designed to revise its decisions as new evidence arrives. Reported savings also account for the cost of TokenEconomizer’s own model calls.

Saving costs in every repository

We evaluated TokenEconomizer on 277 SWE-Bench tasks from six repositories, using five different AI coding agent and model combinations. For each repository, TokenEconomizer started with no prior session history. 

TokenEconomizer lowered the average cost by 18.4–30.9%, as shown in Figure 3, and saved the most in all 30 repository-combination pairs without performance loss. By contrast, lowering the model’s reasoning effort saved similar or larger amounts but led to substantial performance degradation.

On SWE-Bench, TokenEconomizer lowered mean cost per task by 18.4–30.9% across five agent–model combinations (left), while Pass@1 rose in four of them (right).
Figure 3. On SWE-Bench, TokenEconomizer lowered mean cost per task by 18.4–30.9% across five agent–model combinations (left), while Pass@1 rose in four of them (right). 

Savings grow as experience accumulates. Early tasks can cost more while TokenEconomizer learns, but cumulative costs fell below the default in every repository by the sixth task, shown inFigure 4. Each mechanism contributes savings on its own, and combining all three delivers the largest reduction.

Cumulative cost change relative to default over each repository’s tasks, using Claude Code. Starting with no history, every repository becomes cheaper by the sixth task.
Figure 4. Cumulative cost change relative to default over each repository’s tasks, using Claude Code. Starting with no history, every repository becomes cheaper by the sixth task. 

The savings also carried over to sequential web development, with costs 16.0–21.2% lower on Web-Bench. On security auditing, where every task is in a new repository, costs were 6.4–19.6% lower.

Looking ahead

TokenEconomizer opens a path toward coding agents that learn not only how to solve tasks, but also how to spend computation more effectively. Its lightweight controller shows how experience can improve efficiency without changing the underlying agent or model.

The open-source plugin makes these directions accessible to developers and researchers. It is designed to run alongside existing AI coding agents without modifying their underlying models, requires no API key of its own, and keeps what it learns on the developer’s machine. We invite the community to use it as a starting point for exploring agents that become more resourceful with experience, adapting how they work, as well as what they know. 

Read the paper

Access the code

View the project

Opens in a new tab

The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

来源:Microsoft Research · microsoft.com