这是本节的多页打印视图。 点击此处打印.

返回本页常规视图.

2026年

AI Agent 2026年的资料

1 - [产品]Claude Tag

Claude 新推出的 Tag 产品

1.1 - 概述

Claude Tag 概述

基本信息

  • 产品名：Claude Tag
  • 发布：2026 年 6 月 23 日（公开 Beta）
  • 厂商：Anthropic
  • 形态：Slack 频道里的共享 AI 队友；在频道中 @Claude 委派任务
  • 底层模型：默认 Opus 4.8（频道内可切换）
  • 可用计划：Claude Team / Enterprise（Anthropic 第一方服务）。Free / Pro / Max 以及 Bedrock、Vertex 等第三方部署不可用
  • 取代：旧版 Claude in Slack（含 Claude Code in Slack）。Slack 侧已于 2026 年 8 月 3 日切到 Claude Tag
  • 官方产品页：https://claude.com/product/tag
  • 管理后台：https://claude.ai/admin-settings/claude-tag
  • 文档总索引（机器可读，含全部 Claude Tag 条目）：https://claude.com/docs/llms.txt

一句话：Claude Tag 把 Claude Code 那套云端沙箱引擎放进团队 Slack，用组织级 Agent Identity（服务账号）干活，而不是借用某个员工的个人权限。

使用门槛（读资料前先记住）

  • 只有 Claude 组织的 Primary Owner / Owner 能完成身份与频道配置；普通 Admin 看得到但配不完
  • 组织必须先开启 Routines，否则 @Claude 会回不可用
  • 启用了 Zero Data Retention（ZDR）的组织不能用（需要存频道记忆与 session 记录）
  • 频道内 Tag 走组织用量余额；私信 DM 走个人 claude.ai 席位
  • Slack 配对：在频道发送 @Claude connect，15 分钟内把配对码贴到管理页

怎么用这份列表

按编号从 1 往下读即可。每一条对应一篇原文，后面可以单独拆成学习笔记。

  • 必读：深入理解产品所必须过一遍
  • 按需：连接指南、单点用例，按自己的工具栈和场景选读，不必全篇啃完
  • 中文二手文放在最后，只作对照，以官网为准

第一阶段：产品是什么

先建立定位：它不是又一个 Slack 聊天机器人，而是多人、异步、带独立身份的团队 Agent。

  1. Introducing Claude Tag（必读）
    Anthropic 发布博文。多人协作、跨天记忆、Ambient 主动跟进、异步执行；内部产品团队约 65% 代码来自内部版 Tag。

  2. Claude in Slack: Tag @Claude in any thread（必读）
    官方产品页。适合先看它对外承诺的能力边界。

  3. 什么是 Claude Tag？（必读，中文）
    帮助中心总述：频道 Tag / DM / 助手面板三套入口，计费、权限层级、记忆与隐私。

  4. What is Claude Tag?（对照）
    上一篇的英文原文，细节有出入时以这篇和官方 docs 为准。

  5. Work with Claude Tag（必读）
    官方文档总览。管理员一次性配置、终端用户如何派活、计费与 spend limit、常见用途入口。

第二阶段：运行机制（深入核心）

这一阶段是后面所有配置和用法的地基。建议按顺序、每篇读完再进下一篇。

  1. How Claude Tag works（必读）
    会话模型：@Claude → 短暂沙箱 → checklist → 回帖 → 空闲释放。频道 session 与线程 session 的区别，以及和 Cowork / Claude Code 的对比表。

  2. Glossary
    先扫一遍术语：agent identity、Access bundle、scope、Agent Proxy、routine、channel memory、session。读后续文档时回查。

  3. How agent identity works（必读）
    频道里 Claude 用自己的服务账号，不是“替发消息的人行事”。频道 vs DM 的身份分叉、GitHub App 署名。

  4. Agent identity: a new access model for autonomous, team-wide AI（必读）
    概念博文（Noah Zweben，Claude Code 团队）。为什么 “act as the user” 在异步、多驾驶员场景会崩；公开频道共享工作区身份、私有频道隔离；未来 JIT 凭证。

  5. Security and data handling（必读）
    沙箱不持有凭证；凭证在网络边界注入；出站主机白名单；频道成员共享该频道 bundle 的能力。

  6. Data lifecycle and deletion（必读）
    Anthropic 侧存什么、存多久；断开工作区、卸载应用、删频道、离开组织分别发生什么。Slack 里删消息删不掉 session transcript。

  7. Claude Tag settings map
    管理页、用量页、频道内 Configure、个人 connector（仅 DM）分别管什么。

  8. Claude Tag for Claude Code users
    已会 Claude Code 的人必看：哪些配置能迁过来，哪些必须改由管理员配；Slack 线程如何映射到 session。

第三阶段：管理员配置

读完身份模型再看落地。组织里如果暂时没有 Owner 权限，仍建议通读 14–22，知道生产上要配哪些开关。

  1. Set up Claude Tag（必读）
    四步：配对 Slack → 接工具 → 设花费上限 → 私有频道试跑。含前置条件与常见失败。

  2. What the Claude Slack app can access
    写给批准 Slack 安装的人：OAuth scope、能读能发什么、安装本身不授予外部系统权限。

  3. Give Claude access to your tools（必读）
    Access bundle：专用服务账号、先接什么、allowed websites 如何限制可达范围。

  4. Configure GitHub access
    Claude 自己的 GitHub 身份、仓库授权、克隆进沙箱、依赖安装、GitHub Actions。写代码场景优先读。

  5. Configure GitLab access
    与 GitHub 对照，自建 GitLab 时读。

  6. Configure per-channel access（必读）
    bundle 绑到组织 / 工作区 / 单个频道；继承与重叠规则。

  7. Restrict where Claude Tag operates（必读）
    谁能调用、guest 频道、按频道开关、RBAC（Enterprise）、安静或移除。

  8. Manage workspaces and versions
    多个 Slack 工作区、Enterprise Grid、每频道 Off / Legacy / New。

  9. Set a spend limit（必读）
    组织用量余额、频道上限、75%/95% 告警；超限会拒绝任务，不会悄悄截断。

  10. Customize Claude Tag
    按 scope 定制，不是按人。哪些在 claude.ai 配，哪些可在频道里改。

  11. Set up a skills repository Claude can update
    组织 skills 放 git 并自动同步；可授权 Claude 给自己的 skills 开 PR。

  12. Review what Claude Tag has done
    审计页 + 各连接系统自己的日志；如何把一次动作追回 Slack 线程。

  13. Network requirements
    出站 IP、仅 HTTP/HTTPS；和企业防火墙 allowlist 的关系。

  14. Migrate from the earlier Claude in Slack
    身份从“每人自己的账号”变成“组织一个身份”；数据不迁移；频道可钉在 Legacy。

  15. Troubleshoot Claude Tag setup
    配置阶段报错手册。用到再查也可以。

连接指南（按需，不必按号全读）

先读总览，再只打开自己栈里有的服务。自定义 MCP 适合没有预设的内部系统。

  1. Per-service connection guides
  2. Connect a service that isn’t in the list（自定义 / MCP）
  3. Asana
  4. Jira and Confluence
  5. BigQuery
  6. Datadog
  7. GitLab API
  8. Gong
  9. Google Drive, Calendar, and Gmail
  10. HubSpot
  11. Linear
  12. Notion
  13. PagerDuty
  14. Salesforce
  15. Sentry
  16. Snowflake
  17. Stripe
  18. Vercel

第四阶段：作为频道成员怎么用

管理员配完之后，日常就靠这一组。建议 47–54 全读，用例库按自己工作选。

  1. Get started（必读）
    如何确认频道已开通、第一条消息、Claude 能读什么、DM 为何不同。

  2. Commands Claude Tag understands（必读）
    @Claude !restart、!routines 等 bang 命令；卡住或上下文跑偏时用。

  3. Control when Claude Tag responds
    没人 @ 它时会不会回；线程/频道静音。觉得它话太多或突然哑巴时读。

  4. Good habits for working with Claude Tag（必读）
    写清完成定义、选对频道、长任务尽早把产物推到 GitHub/文档，因为沙箱会释放。

  5. Set up routines（必读）
    定时任务、盯频道、订 PR；列出/暂停站岗工作。Ambient 行为的操作面。

  6. What Claude Tag remembers（必读）
    记忆属于频道不是个人；公开频道写入工作区记忆，私有频道隔离。如何查看和纠正。

  7. Choose the model Claude Tag uses
    线程内切换、频道默认、DM 选模型。

  8. Prompt library
    可粘贴的第一句、转发交接、频道规则、记忆检查、中途转向。

  9. Troubleshoot Claude Tag in channels and DMs
    无回复、排队失败、丢失工作、缺连接、链接被拦。用到再查。

用例库（按需）

先读总库，再按场景点开单篇。每篇都带可粘贴 prompt 和所需连接。

  1. Use case library
  2. Triage requests
  3. Catch up
  4. Work from your own channel
  5. Turn threads into docs and tickets
  6. Track projects and chase approvals
  7. Answer data questions
  8. Find answers in your docs
  9. Review documents against a checklist
  10. Pull deal and account state
  11. Claude Tag for marketing teams
  12. Watch monitors and alerts
  13. Fix bugs
  14. Work with your GitHub repositories

第五阶段：内部案例与业界评论

官方 docs 读完后，用案例建立“团队里到底怎么用”的直觉。

  1. How Anthropic employees use Claude Tag（必读）
    Anthropic 内部：客诉/事故汇总、法务审素材、数据自助查询等。

  2. Self-service data analytics in Slack
    内部如何把数仓问答放到共享频道。若链接变更，从上一篇文末的 related posts 进入。

  3. Anthropic releases Claude Tag, a virtual employee that works within Slack
    Fortune 报道。Cat Wu 对“相对 Claude Code / Cowork 的多人产品”表述；和 Slackbot、Agentforce 的竞品语境。

  4. Anthropic and Salesforce Announce New Claude to Slack Integration
    Salesforce 侧对 ambient colleague 的解读。

附录：中文二手整理（对照用，不作为依据）

官方与帮助中心读完后，如需中文导读可看这些，细节冲突以 1–69 为准。

  1. Claude Tag：Anthropic 把 AI 队友放进了你的 Slack
  2. Slack 配置 Claude Tag：4 步上手、ambient 模式
  3. Claude Tag 完整教程（OSCHINA）

建议的精读顺序（压缩版）

时间紧时，先按这条最短路径走完再回头补按需篇：

1 → 3 → 5 → 6 → 8 → 9 → 10 → 14 → 16 → 19 → 20 → 22 → 47 → 50 → 51 → 52 → 56 → 70

最短路径过完，对身份模型、沙箱生命周期、管理员四步、日常派活和内部案例就有完整骨架；其余编号按工具栈和场景插入即可。

1.2 - 信息和介绍

Claude Tag 信息和介绍

本栏按资料列表顺序，存放原文的中英对照。完整编号清单见 概述。

已完成：

  1. Introducing Claude Tag（发布博文）
  2. Tag Claude in Slack（产品页） 3–4. What is Claude Tag?（帮助中心；中英官方合并为一篇）
  3. Work with Claude Tag（文档总览）

编号 6–13（运行机制）见 机制。

1.2.1 - Introducing Claude Tag

Anthropic 发布博文 Introducing Claude Tag 的中英对照

来源： Introducing Claude Tag
日期： 2026 年 6 月 23 日
分类： Product

Introducing Claude Tag / 介绍 Claude Tag

Claude Tag is a new way for teams to work with Claude.

Claude Tag 是团队与 Claude 协作的一种新方式。

We’re starting on Slack, which Claude can join as a team member. Grant Claude access to selected channels, and connect it to whichever tools, data—and even codebases—you choose. Then, anyone in the channel can tag @Claude in, and delegate tasks to it while they focus on other work. Claude builds context by remembering relevant information from the channels it’s in, and can plan out tasks to complete in the future.

我们先从 Slack 开始——Claude 可以像一名团队成员一样加入其中。向选定的频道授予 Claude 访问权限，并把它连接到你选择的任意工具、数据，乃至代码库。之后，频道里的任何人都可以 @Claude，把任务委派给它，自己去忙别的事。Claude 会记住它所在频道中的相关信息来建立上下文，并能规划将来要完成的任务。

We see Claude Tag as the beginning of an evolution of Claude Code: it makes the model even more proactive, and it works better with a full team. Tagging @Claude is now one of the main ways we get things done at Anthropic. Today, 65% of our product team’s code is created by our internal version of Claude Tag. The same pattern is now spreading well beyond engineering—we’re tagging Claude to chase down product metrics and data, work through support tickets, or even help find the root cause of tricky bugs.

我们把 Claude Tag 视为 Claude Code 演化的开端：它让模型更加主动，也更适合与整个团队一起工作。在 Anthropic 内部，@Claude 已经成为我们推进工作的主要方式之一。目前，产品团队 65% 的代码由内部版 Claude Tag 产出。同一模式正在超出工程范围扩散——我们会 @Claude 去追查产品指标与数据、处理支持工单，甚至帮忙定位棘手缺陷的根因。

We’re launching Claude Tag on Slack, since it’s a natural home for collaborative work between teams and AI, and where much of Anthropic’s day-to-day work already happens. It’s available today in beta for Claude Enterprise and Team customers. Our goal is to expand where it’s available more widely, so that teams can tag @Claude in the many other places they work.

我们选择在 Slack 上线 Claude Tag，因为它天然适合团队与 AI 的协作，而 Anthropic 日常工作的很大一部分也已经发生在这里。即日起，Claude Enterprise 与 Team 客户可以以 beta 使用。我们的目标是把它扩展到更多地方，让团队能在日常工作的许多其他场景里 @Claude。

Working with @Claude / 与 @Claude 一起工作

If you’ve worked with Claude Code or Cowork before, Claude Tag will feel familiar. Tag @Claude with a request in simple terms and it’ll break its task down into stages and then work through them in turn, using the tools it has access to. Once it’s done, it’ll respond in a Slack thread with what it’s created.

如果你用过 Claude Code 或 Cowork，Claude Tag 会有熟悉感。用简单的话向 @Claude 提出请求，它会把任务拆成若干阶段，再依次利用它能访问的工具推进。完成后，它会在 Slack 线程里回复它所产出的内容。

But tagging Claude comes with a few new advantages:

不过，@Claude 还带来几项新的优势：

@Claude is multiplayer. Within a given Slack channel, there’s one Claude that interacts with everyone. This means that anyone can see what it’s working on, and can pick up the conversation from where the last person left off. This makes tagging Claude very different from working within a single chat or for a single task—it’s much more like interacting collaboratively with a teammate.

@Claude 是多人的。 在一个给定的 Slack 频道里，只有一个与所有人互动的 Claude。这意味着任何人都能看到它正在做什么，也可以从上一任停下的地方接续对话。这让 @Claude 与「单独开一个聊天」或「只做一件任务」非常不同——它更像是在和一位队友协作。

@Claude learns over time. As Claude follows along with its channel, it builds more context about the work. This means that users don’t need to explain things to it from scratch over and over again. And Claude can even automatically learn from other Slack channels and data sources, if it’s granted permission. (It doesn’t report from private channels.) This gives it the tacit knowledge necessary for it to provide the best possible work.

@Claude 会随时间学习。 随着 Claude 跟进它所在的频道，它会积累关于这项工作的更多上下文。因此用户不必一遍又一遍地从零解释。若获得授权，Claude 甚至还能自动从其他 Slack 频道和数据源学习。（它不会从私有频道向外汇报。）这为它提供了把工作做到最好所需的默会知识。

@Claude takes initiative. If “ambient” behavior is enabled, Claude will proactively keep you updated about whatever it thinks you might need to know. It’ll flag relevant information from across the channels it’s in and the tools it’s connected to, and follow up on threads or tasks that have gone quiet without being resolved.

@Claude 会主动作为。 若启用了 “ambient”（主动在场）行为，Claude 会主动把它认为你可能需要知道的事情更新给你。它会从所在频道和已连接的工具中标出相关信息，并跟进那些陷入安静、却尚未解决的线程或任务。

@Claude works asynchronously. Set Claude a task, and you can focus on your other priorities while it works. It can also schedule tasks for itself, pursuing a project autonomously over hours or days. We’ve found this particularly helpful at Anthropic: we now spend much more of our time delegating tasks to many Claudes in parallel.

@Claude 异步工作。 给 Claude 一项任务后，你可以在它干活时去忙别的优先级。它也可以给自己安排任务，在数小时乃至数天里自主推进一个项目。我们在 Anthropic 内部发现这一点尤其有用：现在我们把更多时间花在把任务并行委派给许多个 Claude 上。

You can also send Claude direct messages: it’ll respond privately, using the personal tools and connectors you’ve set up.

你也可以给 Claude 发私信：它会私下回复，并使用你个人已配置的工具与连接器。

Getting started / 开始使用

We’ve designed Claude Tag with teams and organizations in mind: @Claude’s access to sensitive data and task-specific tools can be very tightly controlled.

我们从一开始就按团队与组织来设计 Claude Tag：@Claude 对敏感数据和任务专用工具的访问可以被非常精细地控制。

To get up and running, system administrators specify which tools and information the model should have access to, in which channels. Think of it as creating separate Claude identities for different uses: everything, including its memories, will stay scoped to the channels defined by the administrators. For example, a model set up for sales work won’t pass on memories to one set up for engineering; nor will it give engineers access to any sales data or tools. More information about provisioning access is available here.

要跑起来，系统管理员需要指定模型应能访问哪些工具与信息、以及在哪些频道中访问。可以把它理解成：为不同用途创建相互分离的 Claude 身份。包括记忆在内的一切，都会被限定在管理员划定的频道范围内。例如，为销售工作配置的模型不会把记忆传给为工程配置的那一个，也不会让工程师拿到任何销售数据或工具。关于如何配置访问权限，见 How agent identity works。

Once permissions are set, everyone can begin tagging right away. Administrators can set limits for token spend (both for the organization and for individual channels), and can view a log of everything that @Claude has done, along with who requested each task.

权限设定完成后，所有人都可以立刻开始 @Claude。管理员可以为 Token 花费设置上限（组织级与单个频道级），并查看 @Claude 做过的全部事项日志，以及每项任务是由谁发起的。

If you’re a Claude Enterprise or Team customer, you have access to Claude Tag in beta starting today. To get started, visit here and follow these four steps:

如果你是 Claude Enterprise 或 Team 客户，即日起即可使用 Claude Tag 的 beta。请打开 管理设置，按下面四步操作：

  1. Pair Claude Tag with your Slack workspace
    将 Claude Tag 与你的 Slack 工作区配对

  2. Give Claude access to your tools
    向 Claude 授予你的工具访问权限

  3. Set a limit on your organization’s monthly spend
    为组织设定每月花费上限

  4. Test Claude in a private channel to confirm it works.
    在私有频道中测试 Claude，确认其工作正常。

Claude Tag replaces the existing Claude in Slack app. To migrate, administrators can opt in within 30 days. We’re issuing an introductory launch credit to eligible Enterprise and Team organizations so that the whole company can try it out.

Claude Tag 将取代现有的 Claude in Slack 应用。迁移时，管理员可在 30 天内选择加入。我们会向符合条件的 Enterprise 与 Team 组织发放上线试用额度，方便全公司试用。

Claude Tag works with Opus 4.8. You can read our docs and product page.

Claude Tag 基于 Opus 4.8。你可以阅读我们的 文档 和 产品页。

1.2.2 - Tag Claude in Slack

官方产品页 Claude in Slack: Tag @Claude in any thread 的中英对照

来源： Claude in Slack: Tag @Claude in any thread
状态： Public Beta
文档： Read documentation

Tag Claude in Slack / 在 Slack 里 Tag Claude

@Claude reacts in real time, where the work is happening. Available today for Claude Enterprise and Team in Slack. Tag it in, and it tags you back.

@Claude 在工作发生的地方实时响应。即日起面向 Slack 中的 Claude Enterprise 与 Team 开放。你把它 Tag 进来，它会把你 Tag 回去。

@Claude reads threads, understands full context, and reacts in real time so your team moves forward together. Bring Claude into your channel.

@Claude 会阅读线程、理解完整上下文，并实时响应，让团队一起往前走。把 Claude 请进你的频道。

Available in beta for Claude Enterprise and Team customers in Slack.

目前以 beta 提供给 Slack 中的 Claude Enterprise 与 Team 客户。

Customer quotes / 客户评价

“Claude Tag is our first responder for internal bugs. It reads incoming reports, looks at screenshots of failures, and uses access to Datadog, Linear, and GitHub to weed out user errors, trace root cause, and often draft a fix PR. Our engineers’ first reaction was ‘this is amazing.’” — Aabhas Sharma, CTO

“Claude Tag 是我们内部缺陷的第一响应者。它阅读收到的报告、查看失败截图，并利用对 Datadog、Linear 和 GitHub 的访问，筛掉用户操作失误、追查根因，还经常起草修复 PR。工程师们的第一反应是‘这太厉害了。’” —— Aabhas Sharma，CTO

“Claude Tag turned a summer of conference logistics — the follow-ups and chasing that always fell through the cracks — into an events program that runs itself while I stay on strategy.” — Matthew Scott, Product Marketing

“Claude Tag 把一个夏天的会议后勤——那些总是漏掉的跟进和催办——变成了一套能自己运转的活动体系，而我继续盯策略。” —— Matthew Scott，产品市场

“Claude Tag doesn’t just do the work — it challenges our thinking, provokes better questions, and elevates our judgement rather than replacing it. And we still feel very much in control throughout.” — Simon Mansfield, Director, Enterprise AI

“Claude Tag 不只是把活干了——它挑战我们的想法、逼出更好的问题，并提升我们的判断，而不是取代判断。全程我们仍觉得掌控在自己手里。” —— Simon Mansfield，企业 AI 总监

“There were no skeptics on our team. Everyone leaned in from day one — it felt like Claude Tag in Slack was something they’d been waiting for.” — Darian Bailey, AI Engineer & Tag Admin

“团队里没有怀疑者。从第一天起所有人都扑了上去——感觉 Slack 里的 Claude Tag 是他们一直在等的东西。” —— Darian Bailey，AI 工程师兼 Tag 管理员

“Claude Tag meets our teams where they already work. Slack is the entry point for everything at Gusto, and the fact that Claude Tag is native there is huge for us. We see a lot of potential here.” — Chad Kunsman, AI Developer

“Claude Tag 在团队已经工作的地方与他们会合。在 Gusto，Slack 是一切的入口，Claude Tag 原生在那里对我们意义很大。我们看到很大潜力。” —— Chad Kunsman，AI 开发者

Coming soon to Microsoft Teams / 即将登陆 Microsoft Teams

Tag @Claude in Teams, where it will read the conversation, understand context, and respond in real time.

在 Teams 里 Tag @Claude，它会阅读对话、理解上下文，并实时响应。

How it works / 它如何工作

Tag Claude into any thread in Slack, no matter how long or messy. It reacts to context, decisions, or open questions to help busy teams get more done.

把 Claude Tag 进 Slack 里任意线程，无论多长、多乱。它会对上下文、决策或未决问题作出反应，帮助忙碌的团队多做成事。

Tag it in. @Claude in any thread or channel and it picks up the context everyone already shares. Ask it to run the numbers, file a ticket, or organize a chaotic chat into action items. It reads the room and does the work.

把它 Tag 进来。 在任意线程或频道里 @Claude，它会接上大家已经共享的上下文。让它跑数字、建工单，或把一团乱的聊天整理成待办。它读懂现场，然后干活。

Set a schedule or let it run long tasks. Give @Claude standing instructions and it works without a mention now or in the future. Ask it to watch a channel, run a weekly digest, flag anything urgent, or page the right person.

设定日程，或让它跑长任务。 给 @Claude 常驻指令，它现在或将来都不必再被提及也能工作。让它盯一个频道、做每周摘要、标出紧急事项，或呼叫对该的人。

It jumps in, and can tag you back. @Claude doesn’t wait to be asked. It surfaces the thread that went quiet, posts when the deploy is live, and flags the decision that needs your call.

它会主动介入，也可以把你 Tag 回去。 @Claude 不会干等被叫。它会把冷掉的线程翻上来，在部署上线时发帖，并标出需要你拍板的决策。

How teams use @Claude / 团队如何使用 @Claude

Claude in Slack can share work in the format your team needs, right in the thread.

Slack 里的 Claude 能以团队需要的形式，直接在线程里交出工作成果。

Catch up fast. Prompt: @Claude What got decided here and what's still open?
Get everything you need from a long thread in seconds. @Claude can surface decisions, open questions, and stakeholders, and you see exactly what’s waiting on you.

快速赶上进度。 提示词：@Claude What got decided here and what's still open?
几秒内从长线程里拿到你需要的一切。@Claude 能标出决策、未决问题和相关干系人，你也能看清哪些正等着你。

Pull the numbers. Prompt: @Claude Top 20 enterprise accounts by spend, last 7 and 28 days.
Ask @Claude to query your data and it posts results like metrics, benchmarks, and charts in the channel, so the whole team sees the answer and can keep asking questions right there.

把数字拉出来。 提示词：@Claude Top 20 enterprise accounts by spend, last 7 and 28 days.
让 @Claude 查询数据，它会把指标、基准和图表发到频道里，全团队都能看见答案，并就地继续追问。

Build from the thread. Prompt: @Claude Fix the bug in this thread and open a draft PR.
Turn a bug report into a draft PR from the context in the thread without ever leaving Slack. Get more done without switching tabs.

从线程里直接做出来。 提示词：@Claude Fix the bug in this thread and open a draft PR.
利用线程里已有的上下文，把缺陷报告变成草稿 PR，而不必离开 Slack。少切标签页，多做成事。

Prep before calls. Prompt: @Claude I'm meeting Acme at 2 — what do I need to know?
Walk into every meeting with a full briefing. CRM notes, recent threads, and call history pulled together before you think to ask.

会前准备。 提示词：@Claude I'm meeting Acme at 2 — what do I need to know?
带着完整简报走进每一次会议。CRM 笔记、最近线程和通话记录，在你想到要问之前就已经汇总好。

Watch what matters. Prompt: @Claude Watch this channel, triage what comes in, and work through the backlog. Tag me only when something needs my call.
@Claude can manage work in the channels your team never had bandwidth for: monitoring backlogs, triaging alerts, and surfacing what needs a human to review, so work that keeps getting pushed out can finally be completed.

盯住真正重要的事。 提示词：@Claude Watch this channel, triage what comes in, and work through the backlog. Tag me only when something needs my call.
@Claude 能管理团队一直腾不出手的频道：监控积压、分诊告警、把需要人审的事项翻上来，那些总被往后推的工作终于能做完。

Core capabilities / 核心能力

Understands your org. The longer Claude works with your team, the more it understands how your organization thinks, how decisions get made, and who owns what.

理解你的组织。 Claude 和团队共事越久，就越理解组织如何思考、决策如何作出、以及谁负责什么。

Memory across threads and days. Context can draw on previous conversations and memory. What happened in Monday’s standup is still there on Thursday, without anyone repeating it.

跨线程、跨天的记忆。 上下文可以调用先前的对话和记忆。周一站会里发生的事，周四还在，不必有人再讲一遍。

A shared resource in your channel. Claude surfaces what matters to the whole channel, not just the person who asked. Everyone sees the work, and anyone can build on it.

频道里的共享资源。 Claude 把对整个频道重要的事翻上来，而不只给提问的那个人。所有人都看见工作，任何人都可以在上面继续做。

Proactive action instead of asking. Set a standing instruction once. Claude watches the channel, runs routines, flags what’s urgent, and tags you back when something needs a decision.

主动行动，而不是等人问。 常驻指令设一次即可。Claude 盯着频道、跑 routines、标出紧急事项，并在需要决策时把你 Tag 回去。

In the work and tools with you. Shape how Claude works in each channel with skills and instructions. Give it access to tools and data to take action on your behalf and post results.

和你一起待在工作与工具里。 用 skills 和指令塑造 Claude 在每个频道里的工作方式。给它工具和数据的访问权，让它代为行动并把结果发出来。

Runs tasks that take days. Hand Claude a complex task and it works while you move on to other projects. It follows up, asks for input, or comes back when it’s done.

能跑耗时数天的任务。 把复杂任务交给 Claude，你转去做别的项目时它继续干。它会跟进、征求意见，或在做完后回来。

@Claude connects to your tools / @Claude 连接到你的工具

Configure any tool with an API using Agent Identity. Claude can query, act, and report back in the thread where the work is happening.

用 Agent Identity 配置任何带 API 的工具。Claude 可以查询、行动，并在工作发生的那条线程里回报。

延伸阅读： Agent identity

Admins stay in control / 管理员保持掌控

Claude acts under its own identity in your systems. Admins configure access once; everyone else just types @Claude.

Claude 在你的系统里以自己的身份行动。管理员配置一次访问权限；其他人只需输入 @Claude。

Customizable identities for granular governance. Claude has its own account in your systems, not a borrowed login. Every credential’s use is logged, so “what did Claude do, and who asked for it?” always has an answer.

可定制身份，用于细粒度治理。 Claude 在你的系统里有自己的账号，而不是借用某个人的登录。每一次凭证使用都会被记录，因此「Claude 做了什么、是谁让它做的」总有答案。

Add it per channel. Decide which tools Claude can reach and read. Confine sensitive connectors to a single private channel or enable broader access for specific tools and resources. Adjust any time from the admin console.

按频道添加。 决定 Claude 能触及和阅读哪些工具。把敏感连接限制在单个私有频道，或为特定工具与资源开放更广的访问。可随时在管理控制台调整。

Related reading on the product page / 产品页上的相关阅读

  • What is Claude Tag? — Learn more about @Claude and what happens to the former Claude for Slack experience.
  • Agent identity: a new security model for autonomous, team-wide AI — Explore our new security model built for agents, not retrofitted from chatbots.
  • Tutorial: Working with @Claude in your workspace — Claude now works alongside your team, under its own account, in the places you already work together. You tag it in the way you’d tag anyone, or have it speak up on its own when there’s something it can help with.

1.2.3 - What is Claude Tag?

帮助中心 What is Claude Tag? 的中英对照

来源（英）： What is Claude Tag?
来源（中）： 什么是 Claude Tag？

What is Claude Tag? / 什么是 Claude Tag？

Claude in Slack switched over to the new Claude Tag experience on August 3, 2026. To integrate Claude and Slack, use Claude Tag instead. Learn how to set up Claude Tag or migrate from the earlier Claude in Slack.

Slack 中的 Claude 已于 2026 年 8 月 3 日切换到新的 Claude Tag 体验。要集成 Claude 和 Slack，请使用 Claude Tag。了解如何 设置 Claude Tag 或 从早期的 Slack 中的 Claude 迁移。

Claude Tag is a new way to work with Claude: tag @Claude into a conversation and it takes on real work, using your organization’s tools and the shared context around it. Claude works under its own identity, builds context by remembering relevant information from the channels it’s in, and can follow up on its own.

Claude Tag 是与 Claude 协作的一种新方式：在对话中标记 @Claude，它会承担真实工作，使用你组织的工具以及周围的共享上下文。Claude 以自己的身份工作，通过记住所在频道的相关信息来建立上下文，并可以自行跟进。

Claude Tag is available on Team and Enterprise plans in beta. Claude Tag works in Slack today.

Claude Tag 在 Team 和 Enterprise 计划中以测试版提供。Claude Tag 目前在 Slack 中可用。

It’s how we’ve brought Claude’s capabilities directly to Slack, bringing AI assistance into your team’s workspace. This integration allows you to work with Claude without leaving Slack through three convenient surfaces:

这是我们将 Claude 的能力直接带进 Slack 的方式，把 AI 协助放进团队工作区。通过三个便捷界面，你可以在不离开 Slack 的情况下与 Claude 协作：

  • Channel tagging: Tag @Claude in any channel to hand it a task, and follow along as it works in the thread.

  • Direct message with Claude: Start a private conversation with @Claude.

  • AI assistant panel: Click the Claude icon in Slack’s AI assistant header to open a panel on the right side of your Slack window, allowing you to access Claude from anywhere in the Slack app.

  • 频道标记： 在任意频道中标记 @Claude 以交付任务，并在线程中跟进它的工作。

  • 与 Claude 的私信： 与 @Claude 开始私人对话。

  • AI 助手面板： 点击 Slack AI 助手标题中的 Claude 图标，在窗口右侧打开面板，从 Slack 应用中的任何位置访问 Claude。

When you tag @Claude in a channel, Claude works through the task while the whole exchange stays visible to everyone in the channel. Everyone in a channel works with the same Claude, so anyone can steer it or pick up where it left off. Claude can also check in on its own, like posting when a job finishes or tagging you when a thread stalls. To learn more, see the Claude Tag overview.

当你在频道中标记 @Claude 时，Claude 会处理任务，整段交互对频道里所有人都可见。频道中的每个人都与同一个 Claude 协作，因此任何人都可以引导它，或从停下的地方接续。Claude 也可以自行检查，例如在工作完成时发帖，或在线程停滞时标记你。了解更多，见 Claude Tag 概述。

In direct messages and the assistant panel, Claude has the capabilities you’ve enabled in your own Claude account, like web search and your connected tools. Channel tagging works differently: Claude acts under your organization’s identity, using the tools and access an admin set up for that channel, and the work is billed to your organization rather than to you.

在私信和助手面板中，Claude 具有你在自己的 Claude 账户里启用的能力，例如网络搜索和已连接的工具。频道标记则不同：Claude 以组织身份行动，使用管理员为该频道配置的工具和访问权限，费用由组织而非你个人承担。

Set up Claude Tag / 设置 Claude Tag

After the Claude app is installed, a Primary Owner or Owner sets up Claude Tag: provision Claude’s identity, connect your organization’s tools and repositories, and choose which channels Claude Tag can work in. People on your team don’t need to set up anything individually once a channel is ready. For the full walkthrough, see the Claude Tag setup guide.

安装 Claude 应用后，由主所有者或所有者设置 Claude Tag：配置 Claude 的身份、连接组织的工具和代码仓库，并选择 Claude Tag 可以在哪些频道中工作。频道就绪后，团队成员无需各自再做设置。完整演练见 Claude Tag 设置指南。

Important: Only a Primary Owner or Owner can set up Claude Tag’s access and channels. The Admin role can’t.

重要： 只有主所有者或所有者可以设置 Claude Tag 的访问权限和频道。Admin 角色无法完成这项设置。

Control who can use Claude Tag / 控制谁可以使用 Claude Tag

In Organization settings > Claude Tag, Member Access has three modes: open to anyone in the Slack workspace, open to any member of your Claude organization, or only members whose role allows it. The third option is role-based access and is available on the Claude Enterprise plan. To restrict by role, set Member Access to “Only members whose role allows it” and grant the “Claude Tag in Slack” capability to a custom role. This setting applies to both channel mentions and direct messages. To set member access, see Restrict where Claude Tag operates.

在组织设置 > Claude Tag 中，成员访问有三种模式：对 Slack 工作区中的任何人开放、对 Claude 组织的任何成员开放，或仅对其角色允许的成员开放。第三种是基于角色的访问，在 Claude Enterprise 计划中可用。要按角色限制，将成员访问设为 “Only members whose role allows it”，并向自定义角色授予 “Claude Tag in Slack” 能力。该设置同时作用于频道提及和私信。设置成员访问见 Restrict where Claude Tag operates。

Manage spend limits for Claude Tag / 管理 Claude Tag 的支出限额

Claude Tag is consumption-based, so spend is based on usage rather than the number of people. As a Primary Owner or Owner, you control it from the usage settings in your admin console.

Claude Tag 按用量计费，支出取决于使用量而非人数。作为主所有者或所有者，你可从管理控制台的用量设置中控制它。

  • Organization-wide limit: a hard cap on total Claude Tag spend across every channel. Spend can’t exceed it.

  • Per-channel limits: set a limit on any individual channel, on top of the organization-wide cap. New channels inherit a default limit.

  • Threshold alerts: admins are notified at 75% and 95% of any limit.

  • Usage analytics: a per-channel spend breakdown lives on the same page.

  • 组织范围限额： 对所有频道 Claude Tag 总支出的硬性上限，支出不能超过它。

  • 按频道限额： 在组织上限之上，为任意单个频道设置限额。新频道继承默认限额。

  • 阈值告警： 任一限额达到 75% 和 95% 时通知管理员。

  • 用量分析： 按频道的支出明细在同一页面。

Note: Work that would go over a limit is declined, never silently cut short. A blocked user can request more from their admin without leaving Slack, and the alert says whether the limit or the available balance caused the block.

注意： 会超出限额的工作会被拒绝，绝不会被静默截断。被拦住的用户可以不离开 Slack 向管理员申请更多额度；告警会说明是限额还是可用余额导致了阻止。

Tagging Claude in a channel is billed to your organization. Direct messages are billed to your own Claude account instead.

在频道中标记 Claude 的费用由组织承担。私信则由你自己的 Claude 账户承担。

To set limits, see Set spend limits for Claude Tag.

设置限额见 Set spend limits for Claude Tag。

Set access and permissions for Claude Tag / 为 Claude Tag 设置访问与权限

You decide what Claude Tag can reach by setting credentials and repository access at three levels. Each level inherits the permissions and memory of the one above it.

你通过在三个层级设置凭证和仓库访问，来决定 Claude Tag 能触及什么。每一级继承上一级的权限和记忆。

  • Organization-wide: credentials and repositories that apply everywhere Claude Tag is installed.

  • Workspace: access that applies to every public channel inside a Slack workspace. Inherits organization-wide permissions and memory.

  • Private channel: extra credentials or repositories on top of what the workspace already grants. Use a private channel to keep sensitive connections to a smaller group. For example, a channel set up for legal work keeps its tools and memory separate from an engineering channel.

  • 组织范围： 适用于安装了 Claude Tag 的所有地方的凭证和仓库。

  • 工作区： 适用于 Slack 工作区内每个公开频道的访问。继承组织范围的权限和记忆。

  • 私有频道： 在工作区已授予内容之上追加的凭证或仓库。用私有频道把敏感连接限制在更小的群体。例如，为法务工作设置的频道，其工具和记忆与工程频道分开。

To configure access, see Claude Tag identity and access.

配置访问见 Claude Tag identity and access。

Channels with guests / 包含访客的频道

Channels that include Slack guests have a separate Allow Claude to respond to guests setting that controls whether Claude responds there, and what access Claude has when it does. Choose Restrict (the default, which blocks Claude from those channels entirely), Channel only (Claude replies, but while a guest is present it runs with channel-only access), or Allow (Claude replies with the full access configured for it, as in any other channel). For the full guide, see Restrict guest channels.

包含 Slack 访客的频道有单独的 “Allow Claude to respond to guests” 设置，用来控制 Claude 是否在那里回复，以及回复时拥有何种访问。可选 Restrict（默认，完全阻止 Claude 进入这些频道）、Channel only（Claude 会回复，但访客在场时只使用频道级访问），或 Allow（Claude 以为其配置的完整访问回复，与其他频道相同）。完整指南见 Restrict guest channels。

Review memory and activity for Claude Tag / 查看 Claude Tag 的记忆与活动

Claude Tag keeps context per channel and per workspace. Admins can view, edit, and delete that memory.

Claude Tag 按频道和工作区保留上下文。管理员可以查看、编辑和删除这些记忆。

An Audit view in Organization settings > Claude Tag > Audit lists every scheduled and one-time task across your organization in addition to all network calls made using Agent Identity. Each action is also traceable in the tool where it happened: posts come from the Claude app in Slack, and commits and pull requests show the Claude GitHub App as the author with a link back to the Slack thread that started them. In any channel, you can ask “@Claude what triggers do you have set up here?” to see and turn off standing work.

组织设置 > Claude Tag > Audit 中的审计视图列出组织内每一项计划任务和一次性任务，以及使用 Agent Identity 发出的所有网络调用。每个动作也能在发生它的工具里追溯：帖子来自 Slack 中的 Claude 应用，提交和拉取请求以 Claude GitHub App 为作者，并带有指回发起它们的 Slack 线程的链接。在任意频道中，可以问 “@Claude what triggers do you have set up here?” 来查看并关闭常驻工作。

Privacy and data / 隐私与数据

Data storage / 数据存储

Your Slack conversations with Claude remain separate from your Claude history, keeping work organized across platforms.

你与 Claude 的 Slack 对话与 Claude 历史记录保持分离，让工作在各平台之间保持有序。

Data visibility / 数据可见性

Conversations initiated in Slack are not visible in your Claude chat history.

在 Slack 中发起的对话，不会出现在 Claude 聊天历史中。

Conversations initiated in the Claude web app are not accessible in Slack.

在 Claude 网页应用中发起的对话，无法在 Slack 中访问。

Each platform maintains separate conversation histories.

每个平台维护各自独立的对话历史。

Data deletion / 数据删除

Conversations are automatically deleted from Claude within 30 days if you disconnect the integration or uninstall the app.

如果断开集成或卸载应用，对话会在 30 天内从 Claude 侧自动删除。

Your conversations in Slack follow your organization’s Slack retention policies.

你在 Slack 中的对话遵循组织的 Slack 保留策略。

Claude Tag memory / Claude Tag 记忆

Claude Tag remembers context to do its work, so channel and workspace memory is retained rather than discarded after each task. Memory and activity respect channel boundaries, and admins can review or delete what Claude remembers. Channel work is attributed to your organization’s Claude identity, while work done in a direct message runs on your own account.

Claude Tag 记住上下文以便完成工作，因此频道和工作区记忆会被保留，而不是在每个任务后丢弃。记忆和活动尊重频道边界，管理员可以审查或删除 Claude 记住的内容。频道工作归属于组织的 Claude 身份，私信中完成的工作则跑在你自己的账户上。

Frequently asked questions / 常见问题

How is Claude Tag different from the Claude in Slack I already use? / Claude Tag 与我已经在用的 Slack 中的 Claude 有何不同？

Claude Tag is the next generation of that experience, in the same place. The familiar ways of working still apply, and Claude can now do more: it remembers context across days, schedules its own follow-ups, checks in proactively, and acts under its own identity. An organization’s Primary Owner or Owner opts your organization in to move over.

Claude Tag 是同一位置上该体验的下一代。熟悉的工作方式仍然适用，而 Claude 现在能做更多：记住跨越数天的上下文、安排自己的跟进、主动检查，并以自己的身份行动。由组织的主所有者或所有者选择让组织迁过去。

Who can set up Claude Tag, and who pays for it? / 谁可以设置 Claude Tag，费用由谁承担？

Only an organization’s Primary Owner or Owner can set up Claude Tag’s access and channels. Tagging Claude in a channel is billed to your organization. Direct messages are billed to the person’s own Claude account instead.

只有组织的主所有者或所有者可以设置 Claude Tag 的访问权限和频道。在频道中标记 Claude 的费用由组织承担。私信则由该人自己的 Claude 账户承担。

1.2.4 - Work with Claude Tag

官方文档 Work with Claude Tag 的中英对照

来源： Work with Claude Tag
状态： Public Beta

Work with Claude Tag / 使用 Claude Tag

Claude Tag puts Claude in your Slack channels with admin-governed access. See what to hand it, how setup works, and where to start as an admin or end user.

Claude Tag 把 Claude 放进 Slack 频道，访问由管理员治理。本文说明可以交给它什么、如何配置，以及管理员和终端用户该从哪里开始。

Tag @Claude in. Get results back in the thread.

把 @Claude Tag 进来。在线程里拿回结果。

Anyone in a channel can tag Claude into a problem and hand it work: reproduce a bug and open a pull request, turn a decision thread into a doc, assemble the state of a project. It posts a checklist in the thread as it goes, and the whole exchange stays visible to the channel.

频道里的任何人都可以把 Claude Tag 进一个问题并交给它工作：复现缺陷并开出拉取请求、把决策线程写成文档、汇总项目现状。它会边做边在线程里发 checklist，整段交互对频道可见。

示例线程 / Example thread（#platform-eng）：

Dana: checkout has felt slow all morning — anyone else seeing it?
Leo: same. @Claude can you investigate? Compare latency against this morning’s deploy and find what’s causing it.
ClaudeAPP: On it. I’ll compare latency before and after the deploy, track down the cause, and report back here.
Done: Pulled p99 latency from Datadog / Diffed deploy 4f2c1 against main / Reproduced the slow query locally / In progress: Opening a pull request with the fix…

Dana：整个上午 checkout 都感觉很慢——还有人遇到吗？
Leo：一样。@Claude 你能查一下吗？对比今天早上那次部署的延迟，找出原因。
ClaudeAPP：收到。我会对比部署前后的延迟，定位原因，然后在这里回报。
完成：从 Datadog 拉取 p99 延迟 / 对比部署 4f2c1 与 main / 在本地复现慢查询 / 进行中：正在打开带修复的拉取请求…

Plans that include Claude Tag / 包含 Claude Tag 的计划

Claude Tag is available on Team and Enterprise plans, on Anthropic’s first-party service. It isn’t available on individual plans (Free, Pro, or Max), or for third-party deployments. To use it, your organization pairs its Slack workspace with its Claude organization; see the setup overview for the full prerequisites.

Claude Tag 在 Anthropic 第一方服务的 Team 和 Enterprise 计划中可用。个人计划（Free、Pro 或 Max）以及第三方部署不可用。要使用它，组织需要将 Slack 工作区与 Claude 组织配对；完整前置条件见 setup overview。

If you’re choosing between Claude products for Slack-shaped work, how Claude Tag differs from Cowork and Claude Code compares them directly: team work in shared channels is Claude Tag; personal work on your own files is Cowork or Claude Code.

如果要在面向 Slack 形态的工作里选择 Claude 产品，how Claude Tag differs from Cowork and Claude Code 有直接对比：共享频道里的团队工作用 Claude Tag；个人文件上的工作用 Cowork 或 Claude Code。

Where Claude Tag runs / Claude Tag 运行在哪里

Claude Tag works in Slack. You interact with it by writing in a Slack channel, thread, or direct message, and it replies there. Mention @Claude in a channel to guarantee it picks the message up.

Claude Tag 在 Slack 中工作。你在频道、线程或私信里写，它就在那里回复。在频道中提及 @Claude，可以确保它收到这条消息。

When Claude works on a task, it runs in an ephemeral sandbox, not on your computer. The sandbox is created when a conversation starts, holds any code or files Claude is working with, and is discarded when the conversation goes idle. See how Claude Tag works for the full lifecycle.

Claude 处理任务时，运行在短暂沙箱里，而不是你的电脑上。对话开始时创建沙箱，容纳 Claude 正在处理的代码或文件，对话空闲后丢弃。完整生命周期见 how Claude Tag works。

You extend what Claude can reach, like your repositories, ticketing systems, data warehouses, and custom tools, through connections, plugins, and skills. An Owner configures these per scope (a channel, a workspace, or the whole organization), separately from any connectors an individual user has set up in their own claude.ai account.

通过 connections、plugins 和 skills，扩展 Claude 能触及的范围，例如仓库、工单系统、数据仓库和自定义工具。由 Owner 按 scope（一个频道、一个工作区或整个组织）配置，与个人用户在自己的 claude.ai 账户里设置的连接器相互独立。

For administrators / 给管理员

For end users / 给终端用户

Billing and spend limits / 计费与支出限额

Adding Claude to Slack doesn’t add a per-seat charge. Channel and thread work is billed by usage instead: it draws from a usage balance, an amount in your organization’s billing currency that an Owner funds. A spend limit caps how much of that balance Claude Tag can use each billing period.

把 Claude 加进 Slack 不会额外按席位收费。频道和线程工作按用量计费：从 Owner 注入的用量余额中扣减（以组织账单货币计）。spend limit 限制每个账期内 Claude Tag 最多能用掉多少余额。

Direct messages don’t draw from this balance. A DM runs on the sender’s own claude.ai account and follows that seat’s usual usage limits, so the organization spend limit doesn’t apply to it.

私信不从这笔余额扣。DM 跑在发送者自己的 claude.ai 账户上，遵循该席位通常的用量限制，因此组织 spend limit 对它不适用。

To learn what your team’s usage costs, run a pilot with a spend limit set and watch the per-channel breakdown on the usage page in your admin settings. Your organization may already have a launch usage credit to run that pilot against before it funds the balance itself.

要了解团队用量成本，先设好 spend limit 做试点，并在管理设置的用量页观察按频道明细。组织可能已有上线试用额度，可在自行注入余额之前先拿它跑试点。

Set a spend limit covers how to fund the balance on each plan, set the limit, and what happens when usage reaches it.

Set a spend limit 说明各计划如何注入余额、如何设限，以及用量触达限额时会发生什么。

Put Claude Tag to work / 让 Claude Tag 开始干活

If Claude Tag is in your channel, you can use it now. (If it isn’t there yet, an Owner in your Claude organization runs setup: see Set up Claude Tag.) Anyone in the channel can hand it work, and channel work bills to the organization, not to you.

如果 Claude Tag 已在你的频道里，现在就可以用。（若还没有，由 Claude 组织中的 Owner 完成设置：见 Set up Claude Tag。）频道里的任何人都可以交给它工作，频道工作记在组织账上，而不是你个人。

What it can reach depends on the channel you’re in, not on who you are. The fastest way to find out is to ask it: @Claude what can you access from this channel? Or, if you’re signed in to your Claude organization, click Configure in the footer of a Claude reply in the channel to see its connections, the external services an admin has connected for that channel. Replies in org-shared channels have no Configure link.

它能触及什么，取决于你所在的频道，而不是你是谁。最快的办法是问它：@Claude what can you access from this channel? 或者，若已登录 Claude 组织，点击频道里 Claude 回复页脚的 Configure，查看管理员为该频道连接的外部服务。组织共享频道里的回复没有 Configure 链接。

The one exception is a DM, where it runs on your own claude.ai account instead of the channel’s setup. Owners can disable DMs organization-wide; see Allow or disable direct messages.

唯一例外是私信：它跑在你自己的 claude.ai 账户上，而不是频道配置。Owner 可以在全组织禁用私信；见 Allow or disable direct messages。

Common uses / 常见用途

The list below covers common ways teams use Claude Tag. Each link opens a guide with the prompts to paste and the connections the task needs.

下面列出团队使用 Claude Tag 的常见方式。每个链接打开一篇指南，含可粘贴的提示词以及该任务所需的连接。

  • Watch monitors and alerts: scheduled dashboard checks, and alerts investigated as they arrive. Needs a monitoring connection like Datadog, Sentry, or PagerDuty.

  • Triage requests: an intake channel where Claude answers what it can, flags duplicates, and routes the rest. Works on Slack content alone.

  • Find answers in your docs: policy and runbook questions answered with the source. Needs a docs connection like Google Drive, Notion, or Confluence.

  • Answer data questions: a plain-language question becomes a warehouse query and a chart. Needs a data warehouse connection.

  • Track projects and chase approvals: standing status digests and follow-ups that run until an approval lands.

  • Turn threads into docs and tickets: a settled discussion becomes the decision doc, the customer reply, or the filed tickets.

  • Fix bugs: a bug reported in the channel comes back as a draft pull request. Needs GitHub.

  • Work from your own channel: scratch questions, digests of channels you don’t follow, and follow-ups on what you said you’d do.

  • 盯监控与告警：定时检查仪表盘，告警一到就调查。需要 Datadog、Sentry 或 PagerDuty 一类监控连接。

  • 分诊请求：接入频道里 Claude 能答的先答、标重复、其余再路由。仅凭 Slack 内容即可工作。

  • 在文档里找答案：政策与 runbook 问题带来源作答。需要 Google Drive、Notion 或 Confluence 一类文档连接。

  • 回答数据问题：自然语言问题变成数仓查询和图表。需要数据仓库连接。

  • 跟踪项目并催办审批：常驻状态摘要，以及一直跟到审批落地的催办。

  • 把线程变成文档和工单：已经谈定的讨论变成决策文档、客户回复或已提交工单。

  • 修缺陷：频道里报的缺陷，回来时是草稿拉取请求。需要 GitHub。

  • 在自己的频道里工作：随手问题、你没在跟的频道摘要，以及对你说过要做之事的跟进。

Get started covers your first message, what you see while Claude works, and how to shape Claude’s behavior in your channel.

Get started 覆盖第一条消息、Claude 工作时你看到什么，以及如何塑造它在频道里的行为。

Set Claude Tag up once for everyone / 为所有人设置一次 Claude Tag

You set up Claude Tag once, at claude.ai/admin-settings/claude-tag, and you must be an Owner in your Claude organization to do it. The setup page at that URL walks you through it:

Claude Tag 只需设置一次，入口是 claude.ai/admin-settings/claude-tag，且必须是 Claude 组织中的 Owner。该页会带你走完：

  • Pair your Slack workspace: send @Claude connect in Slack to get a pairing code, then enter it on the setup page.

  • Connect the services Claude will work in: for each one, such as your issue tracker or data warehouse, create an account for Claude and enter its credential.

  • Grant repositories: choose which repositories the Claude GitHub App can reach.

  • Set a monthly spend limit and launch.

  • 配对 Slack 工作区：在 Slack 发送 @Claude connect 拿到配对码，再填到设置页。

  • 连接 Claude 将使用的服务：对工单系统或数据仓库等每一个服务，为 Claude 创建账号并填入凭证。

  • 授予仓库：选择 Claude GitHub App 能触及哪些仓库。

  • 设定每月花费上限并上线。

Claude Tag starts with no access to your external systems. The services you connect during setup form an Access bundle, the set of tools Claude can reach, attached to the workspace or channels you paired. Once you launch, everyone in a channel Claude is in can use Claude Tag immediately, with no per-user setup.

Claude Tag 一开始不能访问任何外部系统。设置时连接的服务构成 Access bundle——Claude 能触及的工具集合——并绑到你配对的工作区或频道。上线后，Claude 所在频道里的所有人都能立刻使用，无需按人再配。

Set up Claude Tag walks through those steps with what to have ready, what each choice means, and how to verify Claude Tag works once you launch.

Set up Claude Tag 逐步说明要准备什么、每个选择意味着什么，以及上线后如何验证可用。

安全评审见 Security and data handling：安全模型、管理员能限制和不能限制什么、审计轨迹与网络要求。

Where to start with Claude Tag / 从哪里开始

1.3 - 运行机制

Claude Tag 运行机制：会话、身份、安全与设置

本栏按资料列表第二阶段（编号 6–13），存放原文的中英对照。完整编号清单见 概述。栏目结构对齐 介绍。

  1. How Claude Tag works（会话、沙箱、转向、记忆范围）
  2. Glossary（Access bundle、Agent Proxy、scope、routine、channel memory）
  3. How agent identity works
  4. Agent identity: a new access model（Noah Zweben，Claude Code 团队）
  5. Security and data handling
  6. Data lifecycle and deletion
  7. Claude Tag settings map
  8. Claude Tag for Claude Code users

建议按编号顺序读：先扫 Glossary，再进身份模型与安全。

1.3.1 - How Claude Tag works

官方文档 How Claude Tag works 的中英对照：会话、沙箱、转向、记忆范围

来源： How Claude Tag works

How Claude Tag works / Claude Tag 如何工作

Each Claude Tag thread in Slack runs a working session in a sandbox. See how progress shows in the thread, how to steer mid-task, what survives between turns, and how memory is scoped.

每个 Claude Tag 线程都在沙箱里跑一个工作 session。本文说明进度如何出现在线程里、任务中途如何转向、回合之间什么会留下来，以及记忆如何划定范围。

Claude Tag is Claude, working inside your team’s Slack channels. An organization Owner gives it its own accounts to the tools your team uses, so it arrives already able to act, and anyone in a channel can tag it into a problem without setting anything up.

Claude Tag 就是在团队 Slack 频道里工作的 Claude。组织 Owner 给它配好团队所用工具的独立账号，所以它一到就能动手；频道里任何人都可以把它 Tag 进一个问题，无需再配置。

When someone tags Claude in at a channel’s top level, the channel’s own session picks the message up. A task that needs investigation, tools, or a longer exchange gets a working session in a thread under the message, and that thread binds to its own session from then on. Claude works through the task and posts the result back into the conversation. The work runs in an ephemeral cloud sandbox, not on your local machine.

有人在频道顶层 Tag Claude 时，由该频道自己的 session 接住这条消息。需要调查、工具或较长往返的任务，会在该消息下开一个线程工作 session，此后该线程就绑定这个 session。Claude 做完任务，把结果发回对话。工作跑在短暂的云端沙箱里，而不是你的本机。

This page covers:

本文覆盖：

  • Walk through a session: an annotated example thread showing one task end to end

  • Starting a session, tracking progress, and steering mid-thread: what to type, what to watch, and who can redirect

  • Team channels and personal DMs: which surface to use, and how access differs between them

  • Key concepts: agent identity, scheduling, and memory defined

  • Lifecycle of a request: the five-step loop, the checklist, per-channel access, and scheduled tasks

  • Session context and memory: what Claude reads, what survives idle, and what carries across channels

  • 走一遍 session：带注释的示例线程，展示一项任务从头到尾

  • 开始 session、跟踪进度、线程中途转向：写什么、看什么、谁能改方向

  • 团队频道与个人私信：用哪一面，访问如何不同

  • 关键概念：agent identity、调度、记忆

  • 一次请求的生命周期：五步循环、checklist、按频道访问、计划任务

  • Session 上下文与记忆：Claude 读什么、空闲后留下什么、什么跨频道带走

Walk through a Claude Tag session / 走一遍 Claude Tag session

The thread below is one task end to end in Slack: Jordan tags @Claude into #launch-week with a question, a colleague steers mid-thread, and the answer lands in the channel.

下面这段线程是 Slack 里一项任务的全过程：Jordan 在 #launch-week 里 @Claude 提问，同事在线程中途转向，答案落回频道。

示例线程 / Example thread（#launch-week，19 members）：

Jordan 9:02 AM
@Claude where are we on launch prep? Pull together what’s still open from this channel.

ClaudeAPP 9:02 AM
On it. I’ll go through this channel’s open threads and the launch plan.
Done: Read 14 open threads
Done: Cross-checked the launch plan in Drive
Done: Listed who each item is waiting on
Done: Drafted the status summary

Sam 9:06 AM
fold in the vendor quotes from last week’s thread too

ClaudeAPP 9:08 AM
Done. Full status below: eight items closed, three open. The venue contract is the oldest, waiting on legal since the 2nd.

Jordan 9:02 AM
@Claude 发布准备做到哪了？把这个频道里还开着的事项汇总一下。

ClaudeAPP 9:02 AM
收到。我会过一遍本频道未关闭的线程和发布计划。
完成：读了 14 个未关闭线程
完成：对照了 Drive 里的发布计划
完成：列出每项在等谁
完成：起草了状态摘要

Sam 9:06 AM
把上周线程里的供应商报价也折进去

ClaudeAPP 9:08 AM
好了。完整状态如下：八项已关，三项仍开。场馆合同最老，从 2 号起就在等法务。

Each of the five moments in that thread shows a piece of how Claude Tag works:

这条线程里的五个时刻，各对应 Claude Tag 工作方式的一块：

  1. Jordan handed Claude a problem, not a prompt. Typing @Claude in a message that asks for something is what starts a working session.

  2. Claude acknowledged, then went quiet. The “is thinking…” line and the checklist are the progress surface; the silence between 9:02 and 9:06 was the work happening. How the checklist updates

  3. Sam steered Claude without @-mentioning it again. Once a session is active in a thread, it belongs to everyone there. Reply in the thread to steer

  4. The work ran somewhere real, with the channel’s tools. Reading fourteen threads happened in a sandbox built for this thread, and the launch plan came through this channel’s Drive connection. What a session can reach is set per channel. Channel access

  5. The result is in the thread. The whole channel can see it, use it, and build on it. What survives between replies

  6. Jordan 交给 Claude 的是一个问题，不是一段提示词。在一条提出需求的消息里写 @Claude，就会开始工作 session。

  7. Claude 先应答，然后安静。线程底部的 “is thinking…” 和 checklist 才是进度面；9:02 到 9:06 的沉默就是在干活。How the checklist updates

  8. Sam 没有再次 @ 就转向了 Claude。session 一旦在线程里激活，就属于线程里所有人。Reply in the thread to steer

  9. 工作发生在真实环境里，用的是频道的工具。读十四个线程是在为本线程建的沙箱里完成的，发布计划来自该频道的 Drive 连接。session 能触及什么按频道设定。Channel access

  10. 结果在线程里。整个频道都能看见、使用、接着往上做。What survives between replies

The rest of this page takes each piece apart.

后文把每一块拆开讲。

Start a session / 开始一个 session

To start a session, type @Claude in a Slack message and say what you need in that same message (a question to answer, a task to run, a problem to dig into). Jordan’s “@Claude where are we on launch prep?” is the whole move. Anyone in the channel can do it.

要开始 session，在一条 Slack 消息里写 @Claude，并在同一条消息里说出你要什么（要答的问题、要跑的任务、要挖的问题）。Jordan 那句 “@Claude where are we on launch prep?” 就是完整动作。频道里任何人都可以做。

Track Claude’s progress / 跟踪 Claude 的进度

Once your message sends, an “is thinking…” line at the bottom of the thread means Claude picked it up. What happens next depends on the size of the ask. Questions and one-off requests get a direct reply. A longer task, like Jordan’s, gets a checklist instead. How the checklist updates covers how it works and how to read one while it runs.

消息发出后，线程底部出现 “is thinking…” 表示 Claude 已经接住。接下来取决于请求大小。问题和一次性请求会直接回复。更长的任务（如 Jordan 的）会改用 checklist。How the checklist updates 说明它如何工作、运行时如何读。

While a session runs, check in by replying in the same thread. Asking “how’s it going?” in the thread is enough; it reads new replies as it works.

session 运行期间，在同一线程里回复即可过问。问一句 “how’s it going?” 就够；它边干活边读新回复。

Reply in the thread to steer / 在线程里回复以转向

Anyone in the channel can steer a running session by replying in its thread, not just the person who started it. That is what Sam did in the walkthrough. Without re-mentioning @Claude or starting over, he replied in Jordan’s thread, and the session folded his instruction into work already in progress. Add context, redirect the approach, or pick up the result later; a colleague’s thread is yours to continue.

频道里任何人都可以在线程里回复来转向正在跑的 session，不只是发起人。演示里 Sam 就是这样做的。他没有再 @Claude，也没有重开，只是在 Jordan 的线程里回复，session 就把他的指示折进已经在进行的工作。补充上下文、改方向、稍后接结果都可以；同事的线程你也可以继续。

Editing or deleting an earlier message doesn’t steer the session the way a reply does:

编辑或删除更早的消息，并不能像回复那样转向 session：

  • Editing a message: Claude receives a note each time you edit, showing what the message said before the edit and what it says now. Both versions become part of the session’s transcript, so editing a message doesn’t remove the earlier text from what Anthropic stores. An edit doesn’t start a new task or re-address Claude, even if you add @Claude to it.

  • Deleting a reply: Claude gets no notification and keeps the version it already read. Deleting the reply in Slack doesn’t remove it from the session’s transcript.

  • Deleting the thread’s first message: if the thread already has replies, Claude keeps working and the session stays open. If you delete it before anyone has replied, the session closes. Anything Claude already pushed or posted persists, per what survives between replies, and you start a new thread to pick the task back up. Closing the session archives it rather than deleting it, so its transcript, including the message you deleted, stays with the channel’s Claude data until that data is deleted.

  • Correcting course: Claude responds to replies; edits reach it only as notes, and a deleted reply not at all. Say the change in a new reply; the reply is also how you walk back a message it already read.

  • 编辑消息：每次编辑 Claude 都会收到一条说明，展示编辑前和编辑后的文本。两个版本都会进入 session transcript，所以编辑并不能从 Anthropic 存储里抹掉更早的文字。编辑不会开新任务，也不会重新呼叫 Claude，即使你在编辑里加上 @Claude。

  • 删除一条回复：Claude 收不到通知，继续保留它已经读过的版本。在 Slack 里删回复，并不会从 session transcript 里删掉它。

  • 删除线程的第一条消息：若线程已有回复，Claude 继续干活，session 保持打开。若在任何人回复之前删除，session 关闭。Claude 已经推送或发出的内容会按 回复之间留下什么 保留，你需要开新线程把任务捡回来。关闭 session 是归档而不是删除，因此 transcript（包括你删掉的那条消息）会跟着该频道的 Claude 数据，直到那些数据被删除。

  • 纠正方向：Claude 响应的是回复；编辑只作为说明到达，已删的回复则完全不到。把变更写在一条新回复里；撤回它已经读过的消息，同样靠回复。

Team channels and personal DMs / 团队频道与个人私信

Where you message Claude determines whose tools and accounts it uses. In a channel, it acts with the connections an organization admin set for that channel, and the work is attributed to its own accounts. In a DM, the same engine runs with your own claude.ai connectors, and the work is attributed to you, except pull requests, which the Claude GitHub App authors from DMs as well.

你在哪里给 Claude 发消息，决定它用谁的工具和账号。在频道里，它使用组织管理员为该频道配置的 connections，工作记在它自己的账号上。在私信里，同一套引擎跑在你自己的 claude.ai connectors 上，工作记在你名下；例外是拉取请求：从私信开出的 PR 同样由 Claude GitHub App 署名。

Working in… Access Attribution Best for
A channel The channel’s connections, set by an admin The agent’s own accounts Shared work the team should see
A DM Your own claude.ai connectors You Personal tasks using your own data
工作地点 访问 归属 适合
频道 管理员为该频道设置的 connections agent 自己的账号 团队应看见的共享工作
私信 你自己的 claude.ai connectors 你 使用个人数据的私事

The Access column is about external systems. A channel session reaches what the channel was granted, and a DM session reaches what your own account is connected to.

Access 列说的是外部系统。频道 session 能触及频道被授予的范围，私信 session 能触及你自己账户已连接的范围。

Everything below describes channel sessions, where most of the model lives. For the DM side, see direct message channels; for choosing between the two, see pick the right surface.

以下全部描述频道 session——模型的主体在这里。私信一侧见 direct message channels；两者如何选，见 pick the right surface。

How Claude Tag differs from Cowork and Claude Code / Claude Tag 与 Cowork、Claude Code 的差别

Anthropic offers several ways to work with Claude on real tasks; they reach the same kinds of systems but through different mechanisms.

Anthropic 提供多种方式让 Claude 做真实任务；它们能触及同类系统，但机制不同。

Claude Tag Cowork Claude Code
Where Slack channels claude.ai chat Your terminal or IDE
Whose access The team’s: service-account credentials an admin sets per channel Yours: your personal OAuth connectors Yours: your local credentials and filesystem
Who sees the work Everyone in the channel Just you Just you
Best for Shared work the team should see and steer Personal research and drafting Hands-on coding in your own checkout
Claude Tag Cowork Claude Code
在哪里 Slack 频道 claude.ai 聊天 你的终端或 IDE
谁的访问 团队的：管理员按频道设置的服务账号凭证 你的：个人 OAuth connectors 你的：本机凭证和文件系统
谁看见工作 频道里所有人 只有你 只有你
适合 团队应看见并转向的共享工作 个人研究与起草 在自己 checkout 里动手写代码

The short version: team work → Claude Tag; personal work → Cowork or Claude Code. Claude Tag’s connections authenticate the agent itself with service accounts, not any person. Personal connectors apply in a Claude Tag DM, which runs on your own claude.ai account, the same way Cowork does.

短版：团队工作 → Claude Tag；个人工作 → Cowork 或 Claude Code。Claude Tag 的 connections 用服务账号认证的是 agent 自身，不是某个人。个人 connectors 只在 Claude Tag 私信里生效，私信跑在你自己的 claude.ai 账户上，和 Cowork 一样。

Key concepts / 关键概念

Three ideas recur across this page and the rest of these docs.

三个概念在本页和其余文档里反复出现。

  • Agent identity: in channels, Claude acts under its own service accounts that an admin provisions, not as the person who asked. What it can reach is set per channel, so everyone in a channel works with the same access. See How agent identity works.

  • Scheduling and long-running work: a task can run on a schedule, follow a pull request and act when it changes, or keep going across many turns in one thread. The same channel access applies whether a person or a schedule started it. See Set up routines.

  • Memory: what Claude learns in public channels is saved as workspace memory that any channel can use; private channels keep their own. See What Claude remembers.

  • Agent identity： 在频道里，Claude 以管理员配好的自己的服务账号行事，而不是提问的那个人。能触及什么按频道设定，所以频道里所有人用同一套访问。见 How agent identity works。

  • 调度与长时工作： 任务可以按日程跑、跟一个拉取请求并在它变化时行动，或在同一线程里跨许多回合继续。无论是人还是日程启动，都用同一套频道访问。见 Set up routines。

  • 记忆： Claude 在公开频道里学到的内容存为工作区记忆，任意频道都能用；私有频道保留自己的。见 What Claude remembers。

Lifecycle of a request / 一次请求的生命周期

Every session, in any channel, follows the same five-step loop.

任意频道里的每个 session，都走同一套五步循环。

  1. The session starts. Someone tags @Claude with a task that needs a working session, or a scheduled routine runs. At a channel’s top level, the channel’s own session picks the message up and starts the thread’s session.

  2. A sandbox builds. Each thread gets its own isolated working environment.

  3. The working loop runs. Claude works through the task with the channel’s access, editing its checklist in place.

  4. The result lands in the thread. An answer, a doc, a chart, or a pull request.

  5. A quiet period follows. The sandbox is released while the thread persists; a new reply rebuilds it and starts the loop again.

  6. Session 开始。有人用 @Claude Tag 一项需要工作 session 的任务，或一条计划好的 routine 跑起来。在频道顶层，频道自己的 session 接住消息并启动该线程的 session。

  7. 沙箱建起。每个线程有自己的隔离工作环境。

  8. 工作循环运行。Claude 用频道的访问把任务做完，并就地改 checklist。

  9. 结果落在线程里。一个答案、一份文档、一张图，或一个拉取请求。

  10. 随后是安静期。线程还在，沙箱被释放；新回复会重建沙箱并再开循环。

Steps 1 and 2 are starting a session: a message tags Claude in, and a sandbox builds for that thread. Step 3, the working loop, is the checklist below. Steps 4 and 5, the result and the quiet period that follows, are covered in What survives between replies.

第 1、2 步就是开始 session：一条消息把 Claude Tag 进来，并为该线程建沙箱。第 3 步工作循环就是下面的 checklist。第 4、5 步——结果和随后的安静期——见 回复之间留下什么。

Every session runs in an ephemeral sandbox, a real working environment where Claude can read documents, run code, build charts, and open pull requests. Claude clones your GitHub repositories into the sandbox, edits them there, and pushes changes back to GitHub as a branch or pull request. The sandbox runs the same engine that powers Claude Code on the web, Anthropic’s agent for writing and running code, which is why the results are working artifacts rather than chat.

每个 session 都跑在短暂沙箱里——真实的工作环境，Claude 可以读文档、跑代码、做图、开拉取请求。Claude 把 GitHub 仓库克隆进沙箱，在里面改，再作为分支或拉取请求推回 GitHub。沙箱跑的是驱动 Claude Code on the web 的同一套引擎（Anthropic 用于写代码、跑代码的 agent），所以结果是可工作的产物，而不只是聊天。

Two threads in the same channel are two separate sessions with separate sandboxes; sessions don’t share state directly.

同一频道里的两个线程是两个独立 session、两套沙箱；session 之间不直接共享状态。

A channel where Claude works at the top level, outside threads, also carries one session for the channel itself, separate from every thread’s. That session reads the channel’s top-level messages and handles top-level @-mentions, so context carries across separate top-level asks in the same channel; What survives between replies covers how long it lives. For a task that needs investigation, tools, or a longer exchange, it starts a dedicated session in a thread under the message. At the channel’s top level, !restart replaces the channel’s session.

Claude 在频道顶层（线程之外）工作时，频道本身也带着一个 session，与每个线程的 session 分开。那个 session 读频道顶层消息、处理顶层 @提及，因此同一频道里分开的顶层请求可以带着上下文；它活多久见 回复之间留下什么。需要调查、工具或较长往返的任务，会在该消息下的线程里开专用 session。在频道顶层，!restart 会替换频道的 session。

Even with nothing connected, every session starts from the same baseline.

即使什么都没连接，每个 session 也从同一基线开始。

  • It reads its own thread and the channel’s history, including pinned items

  • It searches the workspace’s content

  • It writes and runs code inside the sandbox, which is how a chart comes out of a posted CSV, or a doc out of a long thread, with nothing wired up

  • 读自己的线程和频道历史，包括钉选

  • 搜索工作区内容

  • 在沙箱里写代码、跑代码——所以贴一份 CSV 能出图，长线程能出文档，不必先接线

What Claude posts back / Claude 回帖什么

Claude posts each session’s result in the thread you asked in, choosing the form that fits the work.

Claude 把每个 session 的结果发在你提问的线程里，并选择适合这项工作的形态。

Form What it is When you see it
A reply An answer, list, or summary as a Slack message Questions and short results
A file or chart Attached to the thread the way anyone shares a file Data, images, generated documents
A page kept current Any of the above, edited in place over time Digests, indexes, standing reports
A hosted page A web page published on claude.ai, linked in the thread Dashboards, prototypes, reports
形态 是什么 何时见到
回复 作为 Slack 消息的答案、列表或摘要 问题和短结果
文件或图表 像任何人分享文件一样附在线程上 数据、图像、生成的文档
保持更新的页面 以上任意一种，随时间就地编辑 摘要、索引、常驻报告
托管页面 发布在 claude.ai 上的网页，链接在线程里 仪表盘、原型、报告

A hosted page stays available after the session ends, and Claude updates it when you ask in the thread. Anyone with access to the channel can open it; artifact visibility covers the access model. These are the same artifacts Claude Code publishes, with channel-based access in place of owner-controlled sharing.

托管页面在 session 结束后仍然可用，你在线程里要求时 Claude 会更新它。能进该频道的人都能打开；访问模型见 artifact visibility。这些与 Claude Code 发布的 artifacts 相同，只是用基于频道的访问，代替由所有者控制的分享。

For code work, the result is usually a draft pull request opened under the Claude GitHub App, with the link posted in the thread.

代码工作的结果通常是 Claude GitHub App 名下开出的草稿拉取请求，链接发在线程里。

How the checklist updates / checklist 如何更新

For a longer task, Claude’s first reply is a checklist, a live task list that it edits in place as it goes. Slack does not send notifications when a message is edited, so the thread can look frozen while the list is still moving.

更长的任务里，Claude 的第一条回复是 checklist——一份它边做边就地改的实时任务列表。Slack 不会在消息被编辑时发通知，所以列表还在动时，线程看起来可能像冻住了。

A quiet thread usually means Claude is mid-task, not stuck. Open the thread. Checklist items checked off since you last looked mean the work is moving. In the walkthrough, nothing new arrived in anyone’s notifications between 9:02 and 9:06, while the checklist ticked through fourteen threads of reading. If the work hits a wall, Claude usually says so in a reply rather than going silent. When a thread stays silent well past what the task should need, treat it as a stuck session; see Claude reacted or started thinking, then never replied.

安静的线程通常表示 Claude 正在干活，而不是卡住。打开线程。自你上次看过后又勾掉的 checklist 项，说明工作在推进。演示里，9:02 到 9:06 谁的通知里都没有新消息，同时 checklist 正在勾完十四个线程的阅读。若工作撞墙，Claude 通常会用回复说明，而不是一直沉默。线程安静的时间远超任务该花的时间，就当作卡住的 session；见 Claude reacted or started thinking, then never replied。

Channel access / 频道访问

Connections extend a session’s reach into your own systems. An organization admin attaches access to a scope (the organization, a workspace, or a single channel), so the same request can do more in one channel than in another, and everyone in a given channel works with the same capability.

Connections 把 session 的触及范围扩到你们自己的系统。组织管理员把访问绑到一个 scope（组织、工作区或单个频道），因此同一请求在某个频道能做的比在另一个频道多，而给定频道里所有人能力相同。

A thread locks in its skills, plugins, and custom instructions when it starts, and a running thread keeps that set. Connections and domain rules are enforced on each request, so one an admin adds mid-thread works in a running thread. Claude doesn’t announce a new connection in an existing thread; ask it to use the service by name. A new thread picks up every kind of change, so after a configuration change, start a new top-level thread.

线程启动时锁定其 skills、plugins 和自定义指令，正在跑的线程保持这一套。Connections 和域名规则按每次请求强制执行，因此管理员在线程中途加上的连接，对正在跑的线程也生效。Claude 不会在已有线程里宣布新连接；用服务名请它去用。新线程会接上所有种类的变更，所以改完配置后，开一个新的顶层线程。

How to identify access / 如何确认访问范围

Because access is set per channel rather than per person, the way to find out what a session can reach is to ask it, not to guess from your own permissions.

因为访问按频道而不是按人设定，要知道 session 能触及什么，去问它，而不是从你自己的权限去猜。

  • Ask what Claude can reach. In any channel, @Claude what can you access from this channel? lists its current reach.

  • If Claude cannot reach something, the channel was not granted access. Another channel may have the access, and an organization Owner can add it. How agent identity works covers the model.

  • Personal connectors apply only in DMs. A connection an admin attaches to a channel is separate from a connector on your personal claude.ai account; anything on your own account works in your DMs, not here.

  • 问 Claude 能触及什么。在任意频道发 @Claude what can you access from this channel?，会列出当前范围。

  • 若 Claude 够不到某样东西，是这个频道没被授予访问。另一个频道可能有，组织 Owner 可以加上。模型见 How agent identity works。

  • 个人 connectors 只在私信里生效。管理员绑到频道的 connection，与你个人 claude.ai 账户上的 connector 分开；你自己账户上的东西在私信里可用，这里不行。

One-off and scheduled tasks / 一次性任务与计划任务

A session starts the same way whether a person triggers it or a schedule does. A tagged task runs in its thread’s session, and the sandbox is released once the work finishes. A routine runs the same loop on a schedule, a channel watch, or a repository event, with the channel’s connections, so a recurring digest or watcher gets the same access a typed request would. See set up routines.

无论是人触发还是日程触发，session 的启动方式相同。被 Tag 的任务跑在其线程的 session 里，做完后释放沙箱。routine 按日程、盯频道或仓库事件跑同一套循环，用的是频道的 connections，因此周期性摘要或看守者拿到的访问，与手打请求相同。见 set up routines。

Session context and memory / Session 上下文与记忆

Every session runs the same lifecycle; what varies by place and thread is what it can see, what survives idle, and what it remembers.

每个 session 生命周期相同；随场所和线程变化的是它能看见什么、空闲后留下什么、记住什么。

Conversation context / 对话上下文

A session reads its own thread and its channel. Mentioning @Claude partway into an existing thread gives it a window of the thread’s messages, not the whole thread, with other bots’ replies filtered out. In long threads, restate anything critical.

session 读自己的线程和所在频道。在已有线程中途提及 @Claude，它拿到的是该线程消息的一个窗口，不是整条线程，其他机器人的回复会被滤掉。长线程里，关键内容请再陈述一遍。

Claude works in channels it has been added to, but workspace search can still find messages by keyword from public channels it’s not a member of (the same search any Slack user has). Workspace search is unavailable in channels that include guests. Finding something is broader than being able to act somewhere; to have it participate in a channel directly, invite it with /invite @Claude.

Claude 在已被加入的频道里工作，但工作区搜索仍能按关键词找到它不是成员的公开频道里的消息（与任意 Slack 用户拥有的搜索相同）。包含访客的频道里，工作区搜索不可用。找到某样东西，比能在某处行动更宽；要让它直接参与一个频道，用 /invite @Claude 邀请。

What survives between replies / 回复之间留下什么

A thread is durable, but the sandbox behind it is not. Durable means the thread stays in Slack, and everything Claude read and said in it is kept in the session’s transcript on Anthropic’s side, so the session can pick up where it left off whenever someone replies. Deleting messages or the thread in Slack doesn’t remove them from that transcript. The sandbox is the computer where Claude runs commands and keeps working files for a task. A few minutes after a session finishes its turn, its sandbox is released, and the same session resumes in a fresh one when the next message arrives. A thread’s session keeps resuming this way for as long as people use the thread. Claude replaces it with a new session in two cases.

线程是持久的，背后的沙箱不是。持久意味着线程留在 Slack 里，Claude 在其中读过和说过的一切，都保存在 Anthropic 侧的 session transcript 里，因此有人回复时 session 能从停下的地方接上。在 Slack 里删消息或删线程，并不会从那份 transcript 里抹掉。沙箱是 Claude 跑命令、为任务保留工作文件的那台计算机。session 完成本回合几分钟后，沙箱被释放；下一条消息到达时，同一 session 在全新沙箱里恢复。只要人们还在用这条线程，线程的 session 就会这样一直恢复。Claude 在两种情况下会换成新 session。

When What Claude does
Someone sends @Claude !restart Archives the session right away and starts a fresh one that rereads the thread
The session gets stuck and fails Starts the replacement when the next message arrives in the thread
何时 Claude 做什么
有人发送 @Claude !restart 立刻归档当前 session，开一个重读该线程的新 session
session 卡住并失败 线程里下一条消息到达时启动替换
Survives idle periods
The conversation and its context Yes
Channel memory Yes
Work pushed, posted, or opened as a PR Yes, in the external system
Files that exist only in the sandbox No. Claude recreates them if asked.
空闲后是否留下
对话及其上下文 是
频道记忆 是
已推送、已发帖或已开成 PR 的工作 是，在外部系统里
只存在于沙箱里的文件 否。若被要求，Claude 会重建它们。

For long tasks, ask it to push branches and post drafts as it goes, so deliverables are saved somewhere durable while the work is still running. See Good habits.

长任务请让它边做边推分支、发草稿，这样交付物在工作还在跑时就已经落到持久的地方。见 Good habits。

A running thread isn’t told about configuration changes an admin makes after it started, such as a new connection, plugin, skill, repository grant, or custom instruction. A new thread starts from the scope’s current configuration, so after changing a scope, start a fresh thread to see the change.

正在跑的线程不会被告知管理员在它启动之后做的配置变更，例如新的 connection、plugin、skill、仓库授权或自定义指令。新线程从 scope 的当前配置开始，所以改完 scope 后，开一条新线程才能看到变更。

The channel’s own session, the one that handles top-level messages outside any thread, lives longer than a thread’s. Claude replaces it with a fresh one when a top-level message arrives after about an hour with no top-level activity, when the session is about a day old, or when the channel’s configuration has changed since the session started. Channel memory and the channel’s history are unaffected, so the only visible effect is that Claude no longer carries what the previous session had been working on.

频道自己的 session（处理线程之外的顶层消息）比线程的活得更久。出现以下情况时 Claude 会换成新的：约一小时没有顶层活动后到达一条顶层消息、session 大约一天了，或自 session 启动以来频道配置已变。频道记忆和频道历史不受影响，所以唯一可见的效果是 Claude 不再带着上一个 session 当时在做的事。

Claude also stops reading a channel’s top-level messages once about 100 of them have arrived since it last posted or replied there. An @Claude mention in the channel starts it reading again; see When Claude stops reading a channel. These thresholds are defaults and can change, so treat the numbers as approximate.

自从它上次在该频道发帖或回复以来，大约又到了 100 条顶层消息，Claude 也会停止阅读该频道的顶层消息。在频道里 @Claude 会让它重新开始读；见 When Claude stops reading a channel。这些阈值是默认值，可能变化，数字当作近似。

Channel and workspace memory / 频道记忆与工作区记忆

Memory follows places the same way access does, and it accumulates for the team rather than for any individual.

记忆像访问一样跟随场所，并且为团队累积，而不是为某个人。

Memory from public channels is shared across the workspace, so a decision recorded while working in #launch-week is available when someone asks in #gtm-west. When Claude cites something from a channel you have never used it in, it is reading workspace memory shared from that channel, not a profile of you.

公开频道的记忆在整个工作区共享，所以在 #launch-week 工作时记下的决定，有人在 #gtm-west 提问时也能用。当 Claude 引用一个你从未在其中用过它的频道里的内容时，它读的是从那个频道共享来的工作区记忆，不是你的个人档案。

Private channels read workspace memory while working, and what they save is written to that channel’s own store rather than the workspace store.

私有频道工作时会读工作区记忆，但它们保存的内容写到该频道自己的存储，而不是工作区存储。

To see what it holds, ask @Claude what do you remember about this channel?. Anyone in the channel can correct or remove entries. What Claude Tag remembers covers reading, correcting, and adding to memory.

要看它记了什么，问 @Claude what do you remember about this channel?。频道里任何人都可以纠正或删除条目。What Claude Tag remembers 覆盖读、纠正和向记忆里添加。

The whole model so far fits in one picture, with access set at the scope, memory shared from public channels, work in progress per thread, and DMs outside all of it.

至此整套模型可以收进一张图：访问设在 scope，记忆从公开频道共享，进行中的工作按线程，私信在这一切之外。

DMs are outside this picture; they run on your own account, as covered in Team channels and personal DMs above. Owners can disable DMs organization-wide; see Allow or disable direct messages.

私信在这张图之外；它们跑在你自己的账户上，见上文 团队频道与个人私信。Owner 可以在全组织禁用私信；见 Allow or disable direct messages。

Related resources / 相关资源

1.3.2 - Glossary

官方文档 Glossary 的中英对照：Access bundle、Agent Proxy、scope、routine、channel memory

来源： Glossary

Glossary / 术语表

Claude Tag terms defined in one place. See agent identity, Access bundle, channel manager, connection, scope, Agent Proxy, routine, channel memory, environment, and session.

Claude Tag 术语集中定义。见 agent identity、Access bundle、channel manager、connection、scope、Agent Proxy、routine、channel memory、environment 和 session。

Access bundle

A named set of connections, domain entries, repository access, and rules that an Owner creates for Claude to use. Bundles attach to scopes, and one bundle can serve many scopes. See Give Claude access.

Owner 为 Claude 创建的一套具名组合：connections、域名条目、仓库访问和规则。Bundle 绑到 scope 上，一个 bundle 可以服务许多 scope。见 Give Claude access。

Agent identity

The service accounts Claude acts with: the Claude app in Slack, the Claude GitHub App on code, and the credentials an admin provisions for every other tool. See How agent identity works.

Claude 据以行事的服务账号：Slack 里的 Claude 应用、代码侧的 Claude GitHub App，以及管理员为其他每个工具配好的凭证。见 How agent identity works。

Agent Proxy

The network layer that injects credentials into Claude’s outbound requests. The model and the sandbox are not given the key; Agent Proxy adds the credential at the network boundary when a request matches the rules an admin set. See How agent identity works.

把凭证注入 Claude 出站请求的网络层。模型和沙箱拿不到密钥；当请求匹配管理员设定的规则时，Agent Proxy 在网络边界加上凭证。见 How agent identity works。

Channel manager

A member of your Claude organization whom an Owner has named to set up specific channels. For each channel assigned to them, a channel manager sets the default model, adds repositories their own GitHub account can write to, and manages credentials in the channel’s own bundle, without holding the Owner role. See Delegate channel setup to channel managers.

Claude 组织中的一名成员，由 Owner 指定来配置特定频道。对分配给他们的每个频道，channel manager 设定默认模型、添加其本人 GitHub 账号可写入的仓库，并管理该频道自己的 bundle 里的凭证，而不需要持有 Owner 角色。见 Delegate channel setup to channel managers。

Channel memory

Facts Claude retains while working in a channel, including facts you told it to remember and notes it writes itself. Entries from public channels are shared across the workspace; entries from private channels are saved to that channel’s own store. See What Claude Tag remembers.

Claude 在频道里工作时保留的事实，包括你让它记住的，以及它自己写下的笔记。公开频道的条目在工作区共享；私有频道的条目存到该频道自己的存储。见 What Claude Tag remembers。

The earlier Claude in Slack / 早期的 Slack 中的 Claude

Claude Tag is the second generation of the Claude app in Slack:

Claude Tag 是 Slack 中 Claude 应用的第二代：

Legacy (the earlier Claude in Slack) New (Claude Tag)
Identity Each user links their own claude.ai account One agent identity with org-level service credentials
Sessions Spawned per request One persistent session per thread, shared
Memory None Shared workspace memory plus private-channel memory
Proactive work None Routines and channel watching
Legacy（早期 Slack 中的 Claude） New（Claude Tag）
身份 每个用户链接自己的 claude.ai 账户 一个 agent identity，配组织级服务凭证
Session 按请求生成 每个线程一个持久 session，共享
记忆 无 共享工作区记忆 + 私有频道记忆
主动工作 无 Routines 与盯频道

Your admin chooses which generation answers @Claude in a given channel, so two channels in the same workspace can work differently. See Migrate from the earlier Claude in Slack.

管理员选择给定频道里由哪一代应答 @Claude，因此同一工作区里两个频道可以工作方式不同。见 Migrate from the earlier Claude in Slack。

Connection

A credential for one external service that Claude uses on the channel’s behalf, like a Datadog API key or a GitHub App installation. Connections belong to the agent identity, not to any user, and are grouped into Access bundles by an admin.

Claude 代表频道使用的、针对某一个外部服务的凭证，例如 Datadog API key 或 GitHub App 安装。Connections 属于 agent identity，不属于任何用户，由管理员归入 Access bundles。

A connection is not a connector. A connector belongs to your personal claude.ai account. Claude cannot use your connectors in channels; it uses the channel’s connections. The one exception is a DM, where it uses your own account instead; see how DMs work in this model.

connection 不是 connector。connector 属于你个人的 claude.ai 账户。Claude 不能在频道里用你的 connectors；它用的是频道的 connections。唯一例外是私信，那里它改用你自己的账户；见 how DMs work in this model。

Connector

A tool you add to your own claude.ai account, like Gmail, Google Drive, or a custom MCP server, listed under Customize > Connectors. Connectors are personal; in Slack they apply only in DMs. For the agent-side equivalent that works in channels, see Connection.

你加到自己 claude.ai 账户上的工具，例如 Gmail、Google Drive 或自定义 MCP 服务器，列在 Customize > Connectors。Connectors 是个人的；在 Slack 里只在私信生效。频道里对应的 agent 侧概念见 Connection。

Environment

The sandboxed compute configuration a session runs in, including its network access setting. Environments used here must be scoped to the organization, not to an individual account, because channel sessions run with no user account attached.

session 运行所在的沙箱计算配置，包括其网络访问设置。这里用的 environments 必须划到组织，而不是个人账户，因为频道 session 运行时不附着任何用户账户。

Plugin

A bundle of skills an Owner attaches to an Access bundle or scope, teaching Claude how to use a specific tool or follow a specific process. Anthropic provides plugins for common tools; you can add your own. See Attach plugins.

Owner 绑到 Access bundle 或 scope 上的一组 skills，教 Claude 如何使用特定工具或遵循特定流程。Anthropic 为常见工具提供 plugins；你也可以加自己的。见 Attach plugins。

Routine

A scheduled or run-once task Claude runs on its own, such as a daily digest or a channel watch. Anyone in a channel can ask Claude to set one up, list what’s scheduled, or disable one. Routines run with the channel’s connections, not the creator’s.

Claude 自行运行的计划任务或一次性任务，例如每日摘要或盯频道。频道里任何人都可以请 Claude 设一条、列出已计划的，或禁用一条。Routines 用的是频道的 connections，不是创建者的。

Claude Code also has a feature named routines. Those run under an individual user’s account; Claude Tag routines run under the agent identity.

Claude Code 也有名为 routines 的功能。那些跑在个人用户账户下；Claude Tag 的 routines 跑在 agent identity 下。

Rule

The match conditions Agent Proxy checks against each outbound request. A connection pairs one credential with the rule that decides when to inject it, and a request that matches the rule gets the credential attached at the boundary. A request that nothing allows (no rule, no domain entry, no environment network access setting) is blocked. See Agent Proxy.

Agent Proxy 对每条出站请求检查的匹配条件。一条 connection 把一份凭证与决定何时注入的 rule 配对；匹配该 rule 的请求在边界被附上凭证。没有任何一层允许的请求（没有 rule、没有域名条目、没有 environment 网络访问设置）会被拦截。见 Agent Proxy。

Scope

One of three levels Claude’s settings can target: Default Slack access (the organization-wide root), one Slack workspace, or one channel (public or private). Scopes inherit downward, so a channel gets its workspace’s settings plus any of its own. An Owner attaches Access bundles and instructions at a scope. See Attach the bundle to a scope.

Claude 设置可以瞄准的三个层级之一：Default Slack access（全组织根）、一个 Slack 工作区，或一个频道（公开或私有）。Scope 向下继承，因此一个频道拿到其工作区的设置，再加上它自己的。Owner 在某个 scope 上绑定 Access bundles 和指令。见 Attach the bundle to a scope。

Session

The unit of work behind one conversation. Each Slack thread binds to one persistent session, and anyone in the channel can continue it by replying in the thread. A channel where Claude works at the top level, outside threads, also carries one session for the channel itself, separate from every thread’s. See How Claude Tag works and Restart a stuck or wrong-context session.

一次对话背后的工作单元。每个 Slack 线程绑定一个持久 session，频道里任何人都可以在线程里回复来继续它。Claude 在频道顶层（线程之外）工作时，频道本身也带着一个 session，与每个线程的分开。见 How Claude Tag works 和 Restart a stuck or wrong-context session。

Related resources / 相关资源

1.3.3 - How agent identity works

官方文档 How agent identity works 的中英对照

来源： How agent identity works

How agent identity works / Agent identity 如何工作

Claude Tag acts under its own service accounts in Slack channels, not as you. See how channel access is bounded, how credentials reach it, and why DMs differ.

Claude Tag 在 Slack 频道里以自己的服务账号行事，而不是作为你。本文说明频道访问如何划界、凭证如何到达它，以及私信为何不同。

Claude Tag’s identity depends on where you message it.

Claude Tag 的身份取决于你在哪里给它发消息。

In Slack channels, Claude acts with its own service accounts, rather than as a specific user. An organization Owner provisions this identity during setup, so it arrives with its own account in each system it works in: the Claude app in Slack, the Claude GitHub App on GitHub, and a service account in every other connected tool. Actions it takes are attributed to those accounts; for example, posts come from the Claude app and pull requests show the Claude GitHub App as the author.

在 Slack 频道里，Claude 用自己的服务账号行事，而不是某个具体用户。组织 Owner 在设置时配好这套身份，因此它在每个工作系统里都有自己的账号：Slack 里的 Claude 应用、GitHub 上的 Claude GitHub App，以及其他每个已连接工具里的服务账号。它采取的动作记在那些账号上；例如帖子来自 Claude 应用，拉取请求显示 Claude GitHub App 为作者。

In direct messages (DMs) between a user and @Claude, the provisioned identity does not apply. DMs are one-to-one only; group DMs aren’t supported. A DM has no channel to scope it to, so a DM session runs on the individual’s own claude.ai account instead, with their personal connectors. GitHub is the exception in attribution: a pull request opened from a DM is authored by the Claude GitHub App, the same as in channels, though the session can only work with repositories connected on that user’s own account. Owners can disable DMs organization-wide; see Allow or disable direct messages.

用户与 @Claude 之间的私信（DM）不适用这套配好的身份。私信仅一对一；不支持群组私信。私信没有可绑定的频道 scope，因此私信 session 改跑在该个人自己的 claude.ai 账户上，使用其个人 connectors。归属上 GitHub 是例外：从私信开出的拉取请求由 Claude GitHub App 署名，与频道相同，但 session 只能使用该用户自己账户上已连接的仓库。Owner 可以在全组织禁用私信；见 Allow or disable direct messages。

How Claude behaves in channels (its standing instructions, plugins, and channel memory) is configured separately from its identity; see custom instructions, plugins, and memory for more information.

Claude 在频道里如何表现（常驻指令、plugins、频道记忆）与其身份分开配置；详见 custom instructions、plugins 和 memory。

Channel sessions / 频道 session

When Claude works on a channel task, the request moves through three places:

Claude 处理频道任务时，请求经过三处：

  • The ask happens in your Slack workspace, when a user tags Claude to do something or a scheduled task starts.

  • The work Claude does runs in a sandbox, an isolated working environment built for the thread.

  • The agent’s credentials for any additional connections, such as GitHub or a data warehouse, reach those systems to pull the required information. An organization Owner sets up those credentials as part of provisioning the identity.

  • 请求发生在你的 Slack 工作区：用户 Tag Claude 去做某事，或一条计划任务启动。

  • Claude 做的工作跑在沙箱里——为该线程建的隔离工作环境。

  • agent 为额外 connections（如 GitHub 或数据仓库）准备的凭证到达那些系统，拉取所需信息。组织 Owner 在配身份时设置这些凭证。

The diagram below traces one request through this process.

下图追踪一次请求如何走完这个过程。（原文此处有示意图。）

Tag Claude in a channel / 在频道里 Tag Claude

A user asks Claude to chart last week’s signups or fix a deploy test. The task gets a session in a thread under the message.

用户请 Claude 画出上周注册量，或修一次部署测试。任务在该消息下的线程里获得一个 session。

The session sandbox starts / session 沙箱启动

Claude does the work in an isolated environment built for this thread, reading files, writing documents, and running code. The credentials you provision are not placed in the sandbox; they stay in the credential store and are injected at the proxy.

Claude 在为本线程建的隔离环境里干活：读文件、写文档、跑代码。你配好的凭证不放进沙箱；它们留在凭证库，在代理处注入。

The request crosses Agent Proxy / 请求穿过 Agent Proxy

When the work needs something outside the sandbox, like calling the GitHub API or querying a warehouse, the request crosses Agent Proxy, the network boundary between the sandbox and everything else. Agent Proxy checks it against the rules an admin configured, and decides whether it proceeds and what credential, if any, travels with it.

当工作需要沙箱之外的东西（例如调用 GitHub API 或查询仓库）时，请求穿过 Agent Proxy——沙箱与其他一切之间的网络边界。Agent Proxy 对照管理员配置的规则检查它，决定是否放行，以及（若有）带上哪份凭证。

Agent Proxy attaches a credential / Agent Proxy 附上凭证

A matching credential comes from the credential store, where an admin’s connections are kept. Once saved, a credential is never displayed again; Agent Proxy retrieves it only at the moment of injection and attaches it to the request at the boundary, so the model and the sandbox itself are not given the key.

匹配的凭证来自凭证库，管理员的 connections 存放在那里。一旦保存，凭证不再显示；Agent Proxy 只在注入那一刻取出，并在边界把它附到请求上，因此模型和沙箱本身拿不到密钥。

The result posts back, as Claude / 结果以 Claude 的身份回帖

The credentialed request reaches your system, like GitHub or the warehouse, and the result returns to the thread.

带凭证的请求到达你的系统（如 GitHub 或仓库），结果回到线程。

Agent Proxy

For each outbound request from the sandbox, Agent Proxy checks the destination against three allow layers. A request goes through if any one of them allows it; a host that none of them allows is blocked.

对沙箱发出的每条出站请求，Agent Proxy 用三层允许规则检查目的地。任一层面允许，请求就通过；三层都不允许的主机被拦截。

When the destination Result
Matches a connection’s rule, its allowed websites The proxy attaches that connection’s credential and forwards the request. The credential stays at the proxy; the model and sandbox are not given it.
Is on the bundle’s Domains list but matches no connection The proxy forwards the request without a credential.
Is allowed by the network access setting of the environment the scope’s sessions run on The proxy forwards the request without a credential.
Matches none of these The proxy blocks the request.
当目的地 结果
匹配某条 connection 的 rule（其 allowed websites） 代理附上该 connection 的凭证并转发。凭证留在代理处；模型和沙箱拿不到。
在 bundle 的 Domains 列表上，但不匹配任何 connection 代理不带凭证转发。
被该 scope 的 session 所跑 environment 的网络访问设置允许 代理不带凭证转发。
以上都不匹配 代理拦截该请求。

A new environment’s network access level defaults to Trusted access, so a fresh setup can reach a documented set of package registries and developer hosts before an admin has configured anything. The cloud environments documentation lists the covered hosts. To narrow that default, pin an environment with a stricter level, such as No access.

新 environment 的网络访问级别默认为 Trusted access，因此在管理员尚未配置任何东西之前，全新设置就能到达一组已文档化的包注册表和开发者主机。cloud environments 文档列出覆盖的主机。要收紧该默认，钉一个更严的级别，例如 No access。

The same rules apply to code Claude runs in the sandbox, like curl or a fetch call: a request is blocked unless its host is allowed by one of the layers above.

Claude 在沙箱里跑的代码（如 curl 或 fetch）适用同一套规则：除非其主机被上面某一层允许，否则请求被拦截。

Agent Proxy carries HTTP and HTTPS only. A protocol that isn’t HTTP, such as SSH or a database’s native wire protocol, can’t cross the proxy even to an allowed host.

Agent Proxy 只承载 HTTP 和 HTTPS。非 HTTP 的协议，例如 SSH 或数据库的原生线路协议，即使主机在允许名单上也不能穿过代理。

For the endpoints and addresses your network team may need to allowlist, see Network requirements.

网络团队可能需要加入 allowlist 的端点和地址，见 Network requirements。

How a host gets allowed / 主机如何被允许

A host that none of the three layers above allows is blocked, and Claude names the blocked host in the thread so an admin can add it; see Give Claude access to your tools.

上面三层都不允许的主机被拦截，Claude 会在线程里点出被拦的主机，方便管理员添加；见 Give Claude access to your tools。

Web search vs. network requests / 网页搜索 vs. 网络请求

Claude can search the web from a channel without any Domains entry. Web search is Anthropic’s built-in web search tool, which runs on Anthropic’s servers, not code running in the channel’s sandbox.

Claude 可以从频道搜索网页，不需要任何 Domains 条目。网页搜索是 Anthropic 内置的搜索工具，跑在 Anthropic 的服务器上，而不是频道沙箱里的代码。

The sandbox sends nothing new for a search. Search requests travel to Anthropic the same way the session’s model traffic already does, and the searching happens server-side. The Agent Proxy rules don’t apply to web search; fetching a page or calling a service from the sandbox is an outbound network request and follows them.

搜索时沙箱不会新发出任何东西。搜索请求以 session 的模型流量同一路径到达 Anthropic，搜索发生在服务端。Agent Proxy 规则不适用于网页搜索；从沙箱抓取页面或调用服务才是出站网络请求，要走这些规则。

Searching and opening a page are different actions. A search returns content from the pages it matches, which Claude reads and cites, so it can answer from a page that search surfaced. Opening a URL, whether one you pasted or one a search returned, is a fetch from the sandbox, and the host needs an allow layer. That is why Claude can quote a page it found through search and still report that it can’t open the same link.

搜索和打开页面是不同动作。搜索返回所匹配页面的内容，Claude 阅读并引用，因此它能根据搜索捞到的页面作答。打开一个 URL——无论是你粘贴的还是搜索返回的——是从沙箱发起的 fetch，主机需要一层允许。这就是为什么 Claude 可以引用它通过搜索找到的页面，却仍报告打不开同一条链接。

The web search capability setting in your organization’s claude.ai admin settings governs claude.ai chat; it doesn’t govern Claude Tag sessions, in channels or DMs. If Claude reports that it can’t reach a host from a channel, the fix is a domain entry or the scope’s environment, not that setting.

组织 claude.ai 管理设置里的网页搜索能力开关管的是 claude.ai 聊天；不管 Claude Tag 的 session（频道或私信）。若 Claude 报告从频道够不到某个主机，要修的是域名条目或该 scope 的 environment，不是那个开关。

Agent access / Agent 访问

What Claude can reach in a channel comes from the Access bundles an admin attached to that channel’s scope. Anyone in the channel gets the same capability, and the same request can do more in #platform-eng than in a general channel.

Claude 在频道里能触及什么，来自管理员绑到该频道 scope 上的 Access bundles。频道里任何人能力相同；同一请求在 #platform-eng 能做的比在普通频道多。

This design has four consequences.

这一设计有四项后果。

  • Configure once. Everyone in the scope can use it immediately.

  • Predictability. What Claude can do never changes based on who asked.

  • Personal connectors apply in DMs. A shared channel uses only the service-account connections an admin attached, not connectors on anyone’s claude.ai account.

  • Clean audit. Actions in connected tools show up under a service account your security team already knows how to reason about.

  • 配置一次。该 scope 里所有人立刻能用。

  • 可预期。Claude 能做什么从不因提问者是谁而变。

  • 个人 connectors 在私信里生效。共享频道只用管理员绑上的服务账号 connections，不用任何人 claude.ai 账户上的 connectors。

  • 干净审计。已连接工具里的动作出现在安全团队已经知道如何推理的服务账号下。

That service-account identity is also how Claude appears wherever it acts. In Slack, it posts as the Claude app. On GitHub, commits and pull requests show the Claude GitHub App, and pull requests link back to the Slack thread they came from. In every other connected service, actions appear under the service account an admin provisioned, in that service’s audit log.

这套服务账号身份也是 Claude 在各处行动时的露面方式。在 Slack，它以 Claude 应用发帖。在 GitHub，提交和拉取请求显示 Claude GitHub App，拉取请求链回它们来自的 Slack 线程。在其他每个已连接服务里，动作出现在管理员配好的服务账号下，记在该服务自己的审计日志中。

Direct message channels / 私信频道

A DM with Claude works differently from a channel. There is no scope to attach an identity to, so a DM session runs with your own claude.ai account instead, the same way a Claude Code session on the web does, using your own connectors and credentials, with results attributed to you (pull requests excepted; the Claude GitHub App authors those from DMs too). The diagram contrasts with the channel path above; the sandbox is the same engine, but everything around it is yours.

与 Claude 的私信和频道不同。没有可绑定身份的 scope，因此私信 session 改跑在你自己的 claude.ai 账户上，和网页上的 Claude Code session 一样，使用你自己的 connectors 和凭证，结果记在你名下（拉取请求除外；从私信开出的同样由 Claude GitHub App 署名）。示意图与上面的频道路径对照：沙箱是同一套引擎，周围的一切是你的。（原文此处有示意图。）

The table lines up the two paths on the four dimensions that differ.

下表按四个不同维度把两条路径对齐。

In a channel In a DM
Acts as Its own service accounts You
Access The channel’s Access bundles Your personal connectors
Attribution The agent’s accounts, in each tool’s audit log Your name, except pull requests, which the Claude GitHub App authors
Billing The organization Your seat
在频道里 在私信里
作为谁行事 它自己的服务账号 你
访问 频道的 Access bundles 你的个人 connectors
归属 agent 的账号，记在各工具的审计日志 你的名字；拉取请求除外，由 Claude GitHub App 署名
计费 组织 你的席位

Three of those differences are worth spelling out.

其中三项值得展开。

  • Connectors. The connectors on your account are available, including MCP servers you’ve added.

  • Billing. Usage bills to your seat rather than the organization’s service key.

  • Channel-side configuration. It doesn’t follow you in; the agent’s connections and repository grants don’t apply in DMs.

  • Connectors。你账户上的 connectors 可用，包括你加过的 MCP 服务器。

  • 计费。用量记在你的席位上，而不是组织的服务密钥。

  • 频道侧配置。不会跟着你进来；agent 的 connections 和仓库授权在私信里不适用。

DM work runs under your credentials, so most of it is attributed to you and can reach only what your own accounts can. Pull requests are the exception: Claude authors them as the Claude GitHub App from DMs too, so a repository’s history shows the same author either way, while the repositories it can reach are still only the ones connected on your own account.

私信工作跑在你的凭证下，所以大部分记在你名下，也只能触及你自己账号能触及的。拉取请求是例外：从私信开出的同样由 Claude GitHub App 署名，因此仓库历史里作者两边一样，而它能触及的仓库仍然只是你自己账户上已连接的那些。

Use channels for shared work and DMs for personal tasks, or for data you’d rather access under your own authenticated identity than a shared channel credential.

共享工作用频道；个人任务，或你更希望以自己的已认证身份而不是共享频道凭证去访问的数据，用私信。

Claude Tag versus Claude Code in Slack / Claude Tag vs. Slack 里的 Claude Code

A DM with Claude Tag runs under your own account, which is also how Claude Code in Slack works, routing a coding @-mention to a Claude Code session on the web under the requester’s own account. The two can look identical. The table shows how to tell them apart.

Claude Tag 的私信跑在你自己的账户下，这也是 Slack 里 Claude Code 的工作方式：把编码相关的 @提及路由到请求者自己账户下、网页上的 Claude Code session。两者看起来可以一模一样。下表说明如何区分。

Claude Tag in a channel Claude Code in Slack
Runs under The agent identity an admin provisioned Your own Claude account, linked in the Claude app
GitHub The Claude GitHub App; pull requests belong to the app Your GitHub connection on claude.ai/code; pull requests open under your account
Access The Access bundles an admin attached to the channel Your personal connectors
Billing The organization Your seat
频道里的 Claude Tag Slack 里的 Claude Code
跑在谁下面 管理员配好的 agent identity 你在 Claude 应用里链接的自己的 Claude 账户
GitHub Claude GitHub App；拉取请求属于该应用 claude.ai/code 上你的 GitHub 连接；拉取请求开在你账户下
访问 管理员绑到该频道的 Access bundles 你的个人 connectors
计费 组织 你的席位

If @Claude in your workspace opens pull requests as you, you’re seeing Claude Code in Slack, not a Claude Tag session.

如果工作区里 @Claude 以你的身份开拉取请求，你看到的是 Slack 里的 Claude Code，而不是 Claude Tag session。

Related resources / 相关资源

1.3.4 - Agent identity: a new access model

概念博文 Agent identity: a new access model for autonomous, team-wide AI 的中英对照（Noah Zweben）

来源： Agent identity: a new access model for autonomous, team-wide AI
日期： 2026 年 6 月 24 日
作者： Noah Zweben（Claude Code 团队，Member of Technical Staff）
分类： Product
阅读时间： 约 5 分钟

Agent identity in Claude Tag: a new access model for autonomous, team-wide AI / Claude Tag 中的 Agent identity：面向自主、全团队 AI 的新访问模型

How Claude Tag’s agent identity access model works, and best practices for configuring it in your team’s workspace.

Claude Tag 的 agent identity 访问模型如何工作，以及在团队工作区里配置它的实践建议。

For an AI agent to do its best work on a human-agent team, it needs access to the same tools, documents, and context humans have.

要让 AI agent 在人机混编团队里把工作做到最好，它需要能访问人类已有的同一套工具、文档和上下文。

In a “single player” AI experience (where one person chats with one assistant), that’s straightforward: you connect your own accounts and the agent acts on your behalf. But in a “multiplayer” AI experience like Claude Tag, Claude sits in a shared channel alongside many people at once, and it draws on the tools and context that belong to the workspace, rather than any one individual.

在「单人」AI 体验里（一个人与一个助手聊天），这很直接：你连上自己的账号，agent 代表你行事。但在 Claude Tag 这样的「多人」AI 体验里，Claude 同时坐在共享频道里、旁边有许多人，它调用的是属于工作区的工具和上下文，而不是属于某一个人。

To make multiplayer experiences work, Claude needs its own accounts for those tools, set up by an admin and tied to the workspace. We call this access model agent identity.

为了让多人体验成立，Claude 需要这些工具上属于它自己的账号，由管理员配置并绑到工作区。我们把这套访问模型叫作 agent identity。

In this post, we explain how agent identity works, how it moves permissions from per-user to per-channel, and how to scope it well in your own workspace.

本文说明 agent identity 如何工作、它如何把权限从按人改成按频道，以及如何在你自己的工作区里把范围划好。

Why “act as the user” breaks down / 为什么「以用户身份行事」会垮掉

When you use AI as a personal assistant, you can connect platforms like Google Drive, GitHub, and your calendar, and let the model use your access permissions to read and write in them.

当你把 AI 当个人助手用时，可以连接 Google Drive、GitHub 和日历等平台，让模型用你的访问权限去读写。

This model doesn’t work for Claude Tag for two reasons:

这套模型对 Claude Tag 不适用，原因有二：

  • Increasing agent autonomy. The length of a task that an AI agent can reliably complete on its own has been doubling roughly every four months. Agents now schedule their own tasks for later and respond to events long after the person who asked has logged off. While users set up routines that trigger them to act given certain situations, the agent works largely autonomously.

  • Multiplayer teams. Claude Tag places Claude in shared spaces where teams are already working—e.g., a channel where three engineers and a PM are debugging together. But when more than one person is steering, whose permissions apply? There’s no single choice of person that’d be right all of the time. This gives admins the ability to define what an agent can do in Slack independent from the humans involved, and a distinct tracking of what is done in Slack.

  • Agent 自主性在上升。 AI agent 能可靠独立完成的任务长度，大约每四个月翻一倍。Agent 现在会给自己安排稍后的任务，并在提问者早已下线之后仍响应事件。虽然用户会设 routines，在特定情况下触发它们行动，但 agent 大体上是自主工作的。

  • 多人团队。 Claude Tag 把 Claude 放进团队已经在工作的共享空间——例如三个工程师和一个 PM 一起排障的频道。但当不止一个人在转向时，用谁的权限？没有哪一个人选会永远正确。这让管理员能独立于牵涉其中的人，定义 agent 在 Slack 里能做什么，并对 Slack 里发生的事有单独的追踪。

Claude acts as itself / Claude 作为它自己行事

In a channel where Claude Tag is active, Claude isn’t acting on behalf of a single user. It has its own account in each system it touches: it posts in Slack as the Claude app, opens pull requests as the Claude GitHub App, and queries your warehouse under a service account provisioned by an admin.

在启用了 Claude Tag 的频道里，Claude 不是代表某一个用户行事。它在触及的每个系统里都有自己的账号：在 Slack 以 Claude 应用发帖，以 Claude GitHub App 开拉取请求，在管理员配好的服务账号下查询你的数仓。

And because there are no personal user credentials in play, a shared channel can never become a side door into someone’s private documents.

也因为没有个人用户凭证介入，共享频道永远不会变成通往某人私人文档的侧门。

Inheriting permissions / 权限继承

In the agent identity model, admins define an identity—the baseline set of connections and skills Claude holds everywhere—at the workspace level, and every channel inherits it by default. Then, where it makes sense, they can override it at the channel level, such as by granting the engineering channel access to GitHub and the data warehouse, or confining a CRM connection to a single private channel.

在 agent identity 模型里，管理员在工作区级定义一套身份——Claude 在各处持有的 connections 和 skills 基线——每个频道默认继承。然后在合适的地方，可以在频道级覆盖，例如给工程频道 GitHub 和数仓访问，或把 CRM 连接限制在单个私有频道。

In addition to credentials, admins also define:

除了凭证，管理员还定义：

  • Repository access: which repos Claude can read and write to.

  • Connectors: the tools and API keys that Claude uses to do its job. Across an organization, different API keys can connect to the same service at different permission levels (e.g., Claude might be given read-only warehouse access in a general channel, and write access in the data team’s private one).

  • Skills and plugins: folders of instructions, scripts, and resources Claude loads dynamically to improve performance on specialized tasks.

  • Standing instructions: custom instructions and context for each channel.

  • 仓库访问： Claude 能读写哪些仓库。

  • Connectors： Claude 用来干活的工具和 API 密钥。在一个组织里，不同 API 密钥可以以不同权限级别连到同一服务（例如 Claude 在普通频道只有数仓只读，在数据团队私有频道有写权限）。

  • Skills 和 plugins： Claude 动态加载的指令、脚本和资源文件夹，用来提升专项任务表现。

  • 常驻指令： 每个频道的自定义指令和上下文。

Because this model works around distinct Claude identities, revoking the identity ends Claude’s access everywhere that the identity was used. This takes much less effort to manage than auditing individual agent actions across dozens of user accounts.

因为这套模型围绕彼此分离的 Claude 身份运作，撤销该身份就会结束它被使用过的所有地方的访问。这比在几十个用户账户上审计单次 agent 动作省力得多。

How the agent identity model works / Agent identity 模型如何工作

Agent identity replaces the question “what can this user do?” with “what can this agent do in this compartment?” That’s a departure from per-user Access Control Lists: it means that a channel member without direct access to the repo can ask Claude to read that repo, if the channel’s profile grants Claude that permission.

Agent identity 把问题从「这个用户能做什么？」换成「这个 agent 在这个隔间里能做什么？」这偏离了按人的访问控制列表：意味着一个对仓库没有直接访问的频道成员，也可以请 Claude 去读那个仓库——如果该频道的 profile 授予了 Claude 这项权限。

This is unusual, but we think it is a necessary step toward an access model that works for autonomous, multiplayer agents. Below, we sketch out how to think about setting those boundaries.

这不寻常，但我们认为这是迈向适合自主、多人 agent 的访问模型的必要一步。下面勾勒如何思考这些边界。

How identity boundaries work / 身份边界如何工作

Claude Tag creates a distinct identity for each private channel; public channels in a workspace share a workspace-level identity. Claude’s identity in a legal channel can’t reach code that wasn’t granted there, and its identity in an engineering channel can’t read legal documents that weren’t granted there. Memory and access respect those boundaries: what Claude learns in a private channel never appears in the wider workspace.

Claude Tag 为每个私有频道创建独立身份；工作区里的公开频道共享工作区级身份。法务频道里的 Claude 身份够不到未在那里授予的代码，工程频道里的身份也读不到未在那里授予的法务文档。记忆和访问都尊重这些边界：Claude 在私有频道里学到的，永远不会出现在更宽的工作区里。

The identity belongs to the channel, so anyone in it can tag Claude by default, and admins can scope each channel’s profile to the least-privileged member. On Enterprise plans, role-based access control lets admins go further and decide which members can invoke Claude at all, so a channel governs both what the agent can reach and who can ask.

身份属于频道，因此默认频道里任何人都可以 Tag Claude，管理员可以把每个频道的 profile 收到权限最低的成员那一级。Enterprise 计划上，基于角色的访问控制让管理员可以再进一步，决定哪些成员根本能调用 Claude，于是一个频道既治理 agent 能触及什么，也治理谁能提问。

Broad default access to tools and context / 对工具和上下文的宽默认访问

How teams scope Claude’s tool access in Claude Tag: broad, low-risk integrations run in shared channels under an agent identity, while personal or team-specific tools stay in DMs and run as the user.

团队如何在 Claude Tag 里划定 Claude 的工具访问：宽、低风险的集成跑在共享频道里、落在 agent identity 下；个人或团队专用工具留在私信里、以用户身份运行。（原文此处有示意图。）

Running Claude Tag inside Anthropic, we found that its value compounds with tool and context access. Each connected system makes every other one more useful, because Claude can combine context across them—pulling a thread from Slack, a doc from Drive, a ticket from a tracker, and a query from a warehouse into one answer that no single tool could provide.

在 Anthropic 内部跑 Claude Tag 时，我们发现它的价值随工具和上下文访问而叠加。每多连一个系统，其他系统就更有用，因为 Claude 能跨系统组合上下文——从 Slack 抽线程、从 Drive 抽文档、从工单系统抽票、从数仓抽查询，合成任何一个单工具给不出的答案。

The teams that get the most out of Claude are the ones that grant it generous access from the start, and pare access back depending on their organization’s admin preferences. Agent identity gives admins broad enough scope for Claude to do useful cross-system work, with boundaries firm enough that the access never travels somewhere it wasn’t granted. Our advice is to start with a baseline profile in a few channels, read the audit trail, and then extend access where the work justifies it, one deliberate grant at a time.

从 Claude 拿得最多的团队，是一开始就给它宽访问、再按组织管理员偏好往回收的那些。Agent identity 给管理员足够宽的范围，让 Claude 能做有用的跨系统工作，同时边界足够硬，访问不会跑到未被授予的地方。我们的建议是：先在少数频道用基线 profile，读审计轨迹，再在工作能证明其必要的地方扩展访问，一次一次审慎授予。

For organizations that require even more granularity, admins can disable Claude Tag in specific channels. Admins can also apply role-based access controls (RBAC) to limit access to Claude Tag to specific users.

需要更细粒度的组织，管理员可以在特定频道禁用 Claude Tag。管理员也可以用基于角色的访问控制（RBAC）把 Claude Tag 限制给特定用户。

Direct messages / 私信

With Claude Tag, direct messages work differently than in shared channels. DMs run on users’ individual claude.ai accounts—their connectors, credentials, and name on the result. This makes DMs the right place to work with Claude on tasks and with tools that should never live in a channel, like email drafts or software only you have a license for.

在 Claude Tag 里，私信与共享频道不同。私信跑在用户各自的 claude.ai 账户上——他们的 connectors、凭证，以及结果上的名字。因此，那些绝不该活在频道里的任务和工具（例如邮件草稿，或只有你有许可证的软件），私信才是和 Claude 一起做的地方。

Security and audit / 安全与审计

When an admin adds a connection to a channel’s profile, the credential is stored independently and mapped to that channel’s identity, then injected at the network boundary at request time. Outbound traffic to any host an admin hasn’t allowed is blocked outright. On the audit side, every routine, memory write, and network call made with agent credentials is recorded, and because Claude acts under its own service accounts, those actions also land in each connected system’s own logs.

管理员把一条 connection 加到频道 profile 时，凭证被独立存储并映射到该频道身份，然后在请求时于网络边界注入。到管理员未允许的任何主机的出站流量直接拦截。审计侧，用 agent 凭证做出的每条 routine、每次记忆写入、每次网络调用都被记录；又因为 Claude 以自己的服务账号行事，这些动作也会落到每个已连接系统自己的日志里。

What’s next / 下一步

Agent identity is the foundation of Claude Tag’s access model. In the future, we plan to strengthen our Claude Tag’s security offerings to include just-in-time credential grants—so that a user can approve a single sensitive action in the moment without permanently widening the agent’s scope—and an identity-aware overlay for organizations with more complex clearance structures. This will add user-level checks on top of an agent’s scope, so Claude only acts when both the channel’s profile and the requesting user’s own permissions allow it.

Agent identity 是 Claude Tag 访问模型的地基。未来我们计划加强 Claude Tag 的安全能力，包括即时（just-in-time）凭证授予——让用户能当场批准一次敏感动作，而不永久扩大 agent 的范围——以及面向更复杂密级结构的组织的身份感知叠加层。这将在 agent 的 scope 之上加上用户级检查，只有频道 profile 和请求用户本人的权限都允许时，Claude 才行动。

The shift from single player to multiplayer AI in products like Claude Tag makes long-running, team-based work possible. Agent identity ensures that Claude’s access to tools is broad enough to be useful, but scoped enough to be secure at enterprise scale.

像 Claude Tag 这样的产品从单人到多人 AI 的转变，让长时、基于团队的工作成为可能。Agent identity 确保 Claude 对工具的访问宽到有用，又收得足够紧，能在企业规模上安全。

Learn more about Claude Tag.

了解更多：Claude Tag。

1.3.5 - Security and data handling

官方文档 Security and data handling 的中英对照

来源： Security and data handling

Security and data handling / 安全与数据处理

Claude Tag runs in an isolated sandbox that holds no credentials. Covers sandbox isolation, credential storage, network egress, service accounts, isolating credentials between channels and what one channel can reach, who can open a published artifact, and which members can invoke Claude.

Claude Tag 跑在不持有凭证的隔离沙箱里。本文覆盖沙箱隔离、凭证存储、网络出站、服务账号、频道之间如何隔离凭证以及一个频道能触及什么、谁能打开已发布的 artifact、哪些成员能调用 Claude。

In channels, Claude acts under its own service accounts that an Owner provisions. By default it can read and post in Slack channels it’s been added to and search public channels by keyword; it has no access to your external systems until an Owner adds connections. Each connection is scoped to specific channels and workspaces, and the actions Claude takes in connected tools are attributable to its own service accounts.

在频道里，Claude 以 Owner 配好的自己的服务账号行事。默认它能在已被加入的 Slack 频道里读和发，并能按关键词搜索公开频道；在 Owner 加上 connections 之前，它不能访问你们的外部系统。每条 connection 限定到特定频道和工作区，Claude 在已连接工具里采取的动作可追溯到它自己的服务账号。

Every channel request, whether a person typed it or a schedule triggered it, follows the same path: it runs in an isolated sandbox that holds no credentials. In an Anthropic-hosted environment, requests leave that sandbox only through Agent Proxy and reach your systems under the agent’s own accounts. Sessions in a self-hosted environment run on runners inside your network, and Claude can’t use Access bundles in those sessions yet.

每一条频道请求——无论是人打的还是日程触发的——都走同一条路径：跑在不持有凭证的隔离沙箱里。在 Anthropic 托管的 environment 里，请求只通过 Agent Proxy 离开沙箱，并以 agent 自己的账号到达你们的系统。自托管 environment 里的 session 跑在你们网络内的 runner 上，那些 session 目前还不能使用 Access bundles。

DMs run on the user’s own claude.ai account instead and are covered separately on How agent identity works.

私信改跑在用户自己的 claude.ai 账户上，单独见 How agent identity works。

How a request travels / 请求如何走

Each Slack thread runs in its own sandbox. In an Anthropic-hosted environment, every outbound call from that sandbox passes through the same checkpoints.

每个 Slack 线程跑在自己的沙箱里。在 Anthropic 托管的 environment 里，该沙箱的每条出站调用都经过同一组检查点。

Checkpoint The guarantee
The sandbox Holds no credentials
Agent Proxy Injects credentials from the credential store at request time, and blocks traffic to unlisted hosts by default
Your systems See the agent’s own accounts, so its actions there are attributable
检查点 保证
沙箱 不持有凭证
Agent Proxy 在请求时从凭证库注入凭证，默认拦截未列入的主机流量
你们的系统 看到的是 agent 自己的账号，因此那里的动作可归属

Compute and the sandbox / 计算与沙箱

Sessions run in ephemeral sandboxes, the same infrastructure that runs Claude Code on the web. Each Slack thread gets its own sandbox.

Session 跑在短暂沙箱里，与网页上 Claude Code 用的是同一套基础设施。每个 Slack 线程有自己的沙箱。

When a thread goes quiet, its sandbox is released; replying in the thread builds a fresh one. What persists across that release and rebuild:

线程安静后，沙箱被释放；在线程里回复会建一个新的。释放与重建之间留下什么：

  • Persists: The thread, its visible work, and anything pushed to a branch, opened as a pull request, or posted into Slack.

  • Does not persist: Files that existed only inside the sandbox. To keep generated files, ask Claude to push them to a branch or post them in the thread.

  • 留下：线程、可见的工作，以及推到分支、开成拉取请求或发进 Slack 的任何东西。

  • 不留下：只存在于沙箱内部的文件。要保留生成的文件，请 Claude 把它们推到分支或发到线程里。

Claude Tag retains channel memory and session transcripts. Because of that retention, Claude Tag isn’t available to organizations with Zero Data Retention (ZDR) enabled.

Claude Tag 保留频道记忆和 session transcripts。因为这项保留，启用了 Zero Data Retention（ZDR）的组织不能使用 Claude Tag。

Credential storage / 凭证存储

Credentials you provision are kept in a separate credential store, not in the proxy itself. When an outbound request matches a rule, Agent Proxy, the network layer between the sandbox and any external host, retrieves the credential from that store and injects it at the boundary, so the model and the sandbox are not given the key.

你配好的凭证放在单独的凭证库里，不在代理本身。当出站请求匹配一条 rule 时，Agent Proxy（沙箱与任何外部主机之间的网络层）从该库取出凭证并在边界注入，因此模型和沙箱拿不到密钥。

This means:

这意味着：

  • A saved credential is not displayed again. The setup screens are write-only.

  • The credential travels only to the hosts you named when you added the connection.

  • You can narrow the credential further, to one host, one path prefix, or read-only methods, in Add connections.

  • 已保存的凭证不再显示。设置界面是只写的。

  • 凭证只到达你添加 connection 时点名的主机。

  • 你可以在 Add connections 里进一步收窄凭证：一台主机、一个路径前缀，或只读方法。

Network egress / 网络出站

In an Anthropic-hosted environment, outbound traffic from a channel session’s sandbox is default-deny. Requests go only to hosts an allow layer covers, and the layers are a connection’s Allowed websites, the bundle’s Domains tab, and the network access setting of the environment the scope’s sessions run on. A new environment’s default level, Trusted access, already covers a documented set of package registries and developer hosts. See Agent Proxy for what happens to a request under each layer.

在 Anthropic 托管的 environment 里，频道 session 沙箱的出站流量默认拒绝。请求只到达某一允许层覆盖的主机；这些层是 connection 的 Allowed websites、bundle 的 Domains 页，以及该 scope 的 session 所跑 environment 的网络访问设置。新 environment 的默认级别 Trusted access 已经覆盖一组已文档化的包注册表和开发者主机。每一层下请求会怎样，见 Agent Proxy。

Because requests to any other host are blocked, data can only leave the sandbox to hosts an allow layer covers. An admin sets the Allowed websites list on each connection and the Domains tab on each bundle. An admin sets the environment’s network access level, which defaults to Trusted access, from the Cloud environments page in admin settings. See Set allowed websites and Allow a host without a credential.

因为到任何其他主机的请求都被拦截，数据只能离开沙箱到达允许层覆盖的主机。管理员在每条 connection 上设 Allowed websites 列表，在每个 bundle 上设 Domains 页。管理员在管理设置的 Cloud environments 页设定 environment 的网络访问级别，默认为 Trusted access。见 Set allowed websites 和 Allow a host without a credential。

Organizations can opt in to allow-all egress, where a * entry on a bundle’s Domains tab admits requests to any host on the ports that entry lists, still without credentials. Private and internal network addresses and cloud metadata endpoints remain blocked. Allow-all egress is off by default and enabled per organization by Anthropic; see Allow all hosts.

组织可以选用 allow-all egress：bundle 的 Domains 页上一条 * 条目，允许到该条目所列端口上任意主机的请求，仍然不带凭证。私有和内部网络地址以及云元数据端点仍然被拦截。Allow-all egress 默认关闭，由 Anthropic 按组织开启；见 Allow all hosts。

Service accounts / 服务账号

In channels, Claude acts under service credentials of its own, not under the account of the person who tagged it. The Slack surface is the Claude app, code work goes through the Claude GitHub App, and every other connected tool uses a service account an Owner provisions in an Access bundle. See How agent identity works for the full model.

在频道里，Claude 以自己的服务凭证行事，而不是 Tag 它的那个人的账号。Slack 面上是 Claude 应用，代码工作走 Claude GitHub App，其他每个已连接工具使用 Owner 在 Access bundle 里配好的服务账号。完整模型见 How agent identity works。

A connection belongs to that agent identity and is shared by everyone the bundle’s scope covers. Anyone in a channel under that scope can ask Claude to act with the credential, so whatever the connected account can read or write is available to every member of those channels. Connect a dedicated identity you control for each service, such as a claude@yourcompany.example.com seat or a native service account, rather than a personal login. A dedicated account keeps the agent’s actions separately auditable in each tool’s logs and lets you revoke its access without affecting a person; see Create a dedicated account per service.

一条 connection 属于那套 agent identity，并由该 bundle 的 scope 覆盖的所有人共享。该 scope 下频道里的任何人都可以请 Claude 用这份凭证行动，因此已连接账号能读或写的一切，对这些频道的每个成员都可用。为每个服务连接一个你控制的专用身份，例如 claude@yourcompany.example.com 席位或原生服务账号，而不是个人登录。专用账号让 agent 的动作在各工具日志里可单独审计，也让你能撤销其访问而不影响某个人；见 Create a dedicated account per service。

DMs with @Claude run on the user’s own claude.ai account instead, with that user’s personal connectors, and work there is attributed to them, except pull requests, which the Claude GitHub App authors from DMs as well. Personal connectors apply only in DMs, never in channels. Owners can disable DMs organization-wide; see Allow or disable direct messages.

与 @Claude 的私信改跑在用户自己的 claude.ai 账户上，使用该用户的个人 connectors，那里的工作记在他们名下；拉取请求除外，从私信开出的同样由 Claude GitHub App 署名。个人 connectors 只在私信生效，从不在频道。Owner 可以在全组织禁用私信；见 Allow or disable direct messages。

Isolate credentials between channels / 在频道之间隔离凭证

A channel session can use only the Access bundles attached in one of three places:

频道 session 只能使用绑在以下三处之一的 Access bundles：

  • The channel itself. A bundle you attach here applies in that channel only.

  • The channel’s workspace. A bundle you attach here applies in every channel of that workspace.

  • Default Slack access. The organization-wide root; a bundle you attach here applies in every channel of every paired workspace.

  • 频道本身。绑在这里的 bundle 只在该频道生效。

  • 该频道的工作区。绑在这里的 bundle 在该工作区的每个频道生效。

  • Default Slack access。全组织根；绑在这里的 bundle 在每个已配对工作区的每个频道生效。

A bundle attached anywhere else in your organization is invisible to the session, and no request from the session’s sandbox can carry a credential from a bundle outside those three scopes.

绑在组织里其他任何地方的 bundle 对该 session 不可见，session 沙箱发出的请求也不能携带这三层 scope 之外的 bundle 的凭证。

For example, if you attach a bundle holding finance credentials to one private channel, sessions in every other channel run as if that credential doesn’t exist. If you attach the same bundle to a workspace or to Default Slack access instead, every channel beneath it gets that access, so isolation comes from where you attach the bundle, not from the bundle itself.

例如，若你把持有财务凭证的 bundle 绑到一个私有频道，其他每个频道的 session 运行起来就像那份凭证不存在。若你把同一 bundle 绑到工作区或 Default Slack access，它下面的每个频道都拿到这份访问，因此隔离来自你把 bundle 绑在哪里，而不是 bundle 本身。

Confine a credential to one channel in three steps:

把一份凭证限制在一个频道，三步：

  1. Attach its bundle to that channel and nowhere broader.

  2. Keep the channel private. A bundle on a public channel grants its access to anyone who joins.

  3. Check the channel’s Access summary on the Slack tab in admin settings. It shows the access the channel actually gets, including what it inherits from the workspace and Default Slack access.

  4. 把它的 bundle 绑到该频道，不要绑到更宽的地方。

  5. 保持频道私有。公开频道上的 bundle 会把访问授予任何加入的人。

  6. 在管理设置的 Slack 页查看该频道的 Access summary。它显示频道实际拿到的访问，包括从工作区和 Default Slack access 继承的。

Claude doesn’t operate in externally shared channels, so a channel shared with another company never has a session to isolate.

Claude 不在对外共享的频道里运作，因此与另一家公司共享的频道永远不会有需要隔离的 session。

Isolating a credential doesn’t isolate what Claude knows. What it learns in a public channel becomes workspace memory that sessions in the workspace’s other channels can read, and it can search public channels by keyword without being added to them, the same way any workspace member can.

隔离凭证并不能隔离 Claude 知道的东西。它在公开频道里学到的会成为工作区记忆，工作区其他频道的 session 能读；它也可以按关键词搜索公开频道而不必被加入，和任意工作区成员一样。

Artifact visibility / Artifact 可见性

A session can publish an artifact, a web page hosted on claude.ai with the link posted in the thread, and the page stays available after the sandbox is released. Anyone with access to the source Slack channel can open it, which in a public channel covers everyone in the workspace. Someone who opens the link without that access sees a request-access prompt rather than the page. There is no share setting for anyone to change. Updates go through Claude: ask in the Slack thread, or send Claude a comment on the page, which anyone who can post in the channel can do.

session 可以发布 artifact——托管在 claude.ai 上的网页，链接发在线程里——沙箱释放后页面仍然可用。能进源 Slack 频道的人都能打开；公开频道里这覆盖工作区所有人。没有该访问的人打开链接会看到请求访问提示，而不是页面。没有任何人可以改的分享设置。更新走 Claude：在 Slack 线程里要求，或在页面上给 Claude 发评论（能在频道里发帖的人都可以）。

Artifacts you publish from your own Claude Code sessions work differently: they belong to you, and you control who can open them, with sharing options that depend on your plan and organization settings. See the Claude Code artifacts documentation.

你从自己的 Claude Code session 发布的 artifacts 不同：它们属于你，由你控制谁能打开，分享选项取决于你的计划和组织设置。见 Claude Code artifacts 文档。

Member access / 成员访问

By default, anyone in a connected Slack workspace can invoke Claude in channels, with or without a Claude account. An Owner can turn on a restriction toggle to narrow that: on Team plans it limits Claude to people with a Claude account in your organization, and on Enterprise plans it limits Claude to members whose role grants the Claude Tag in Slack capability. See Restrict who can use Claude. The toggle governs DMs as well as channels.

默认情况下，已连接 Slack 工作区里的任何人都可以在频道里调用 Claude，有没有 Claude 账户都行。Owner 可以打开限制开关来收窄：Team 计划上把 Claude 限制给组织里有 Claude 账户的人；Enterprise 计划上限制给角色授予了 Claude Tag in Slack 能力的成员。见 Restrict who can use Claude。该开关同时管私信和频道。

Related resources / 相关资源

1.3.6 - Data lifecycle and deletion

官方文档 Data lifecycle and deletion 的中英对照

来源： Data lifecycle and deletion

Data lifecycle and deletion / 数据生命周期与删除

What Claude Tag stores on Anthropic’s side, how long session transcripts and memory are kept, and what disconnecting a workspace, uninstalling the app, deleting a Slack channel, removing a scope, or leaving the organization does to that data.

Claude Tag 在 Anthropic 侧存什么、session transcripts 和记忆保留多久，以及断开工作区、卸载应用、删除 Slack 频道、移除 scope 或离开组织分别对那些数据做什么。

Claude Tag keeps a record of its work on Anthropic’s side, separate from the messages in your Slack workspace. This page is for the Owner or security reviewer who needs to know what that record contains, how long Anthropic keeps it, and which actions in Slack or in your Claude admin settings delete it. For each action, the tables below say what is deleted and whether Claude keeps responding, because the two don’t always go together.

Claude Tag 在 Anthropic 侧保留其工作的一份记录，与你 Slack 工作区里的消息分开。本页写给需要知道这份记录含什么、Anthropic 保留多久、以及 Slack 或 Claude 管理设置里哪些动作会删除它的 Owner 或安全评审人。对每个动作，下表说明删了什么、Claude 是否继续应答——这两者并不总是一起发生。

What Anthropic stores / Anthropic 存什么

There are two copies of the work Claude does in Slack. The messages, canvases, files, bookmarks, and pins in Slack stay in Slack under your workspace’s own retention settings. Anthropic doesn’t remove them when Claude-side data is deleted, and can’t once the app is uninstalled, so delete those in Slack. On Anthropic’s side, Claude Tag stores the following for each paired workspace.

Claude 在 Slack 里做的工作有两份拷贝。Slack 里的消息、canvas、文件、书签和钉选留在 Slack，遵循你工作区自己的保留设置。Claude 侧数据被删除时 Anthropic 不会移除它们，应用卸载后也做不到，所以那些要在 Slack 里删。在 Anthropic 侧，Claude Tag 为每个已配对工作区存储以下内容。

What What it contains
Session transcripts The record of one thread’s, channel’s, or direct message’s work. A transcript holds the messages Claude was shown in the conversation and who sent them, files attached there, earlier versions of messages that were later edited, what Claude retrieved while working (Slack search results, messages it read in channels it’s in, and data from connected tools), and everything Claude said and did
Memory The notes Claude saves for each channel, the workspace-shared notes from public channels, and separate notes for each direct-message conversation. See What Claude Tag remembers
Routines The scheduled and run-once tasks set up in channels or in direct messages, with their instructions and run history
Scopes and their settings Each workspace and channel scope, with its custom instructions, version setting, and Access bundle bindings
Access bundles The connections and credentials an Owner provisions. Bundles belong to your organization, not to a workspace
Account links The link between each member’s Slack account and their Claude account, and the tokens that link uses
Published artifacts Pages a session published to claude.ai. See Artifact visibility
The app’s installation credential The token the Claude app uses to read and post in your Slack workspace
什么 含什么
Session transcripts 一条线程、一个频道或一则私信的工作记录。Transcript 保存对话里给 Claude 看过的消息及发送者、那里附上的文件、后来被编辑的消息的更早版本、Claude 工作时检索到的内容（Slack 搜索结果、它在所在频道读过的消息、来自已连接工具的数据），以及 Claude 说过和做过的一切
Memory Claude 为每个频道保存的笔记、来自公开频道的工作区共享笔记，以及每则私信对话各自的笔记。见 What Claude Tag remembers
Routines 在频道或私信里设置的计划任务和一次性任务，含指令和运行历史
Scopes 及其设置 每个工作区和频道 scope，含自定义指令、版本设置和 Access bundle 绑定
Access bundles Owner 配好的 connections 和凭证。Bundles 属于组织，不属于某个工作区
Account links 每个成员的 Slack 账户与 Claude 账户之间的链接，以及该链接使用的 token
Published artifacts session 发布到 claude.ai 的页面。见 Artifact visibility
应用的安装凭证 Claude 应用用来在你的 Slack 工作区读和发的 token

Deleting or editing a message or file in Slack doesn’t remove it from a transcript that already includes it. Messages from other people in the conversation, including guests, become part of the transcript the same way and are held as your organization’s data.

在 Slack 里删除或编辑消息或文件，并不会从已经包含它的 transcript 里抹掉。对话里其他人的消息（包括访客）同样进入 transcript，并作为你组织的数据持有。

How long data is kept / 数据保留多久

During the beta there is no automatic retention period for Claude Tag data. Session transcripts, memory, routines, scopes, and account links are kept until one of the actions on this page deletes them. Your organization’s custom data retention setting doesn’t apply to Claude Tag transcripts or memory. It does apply to artifacts a session published, which follow the same retention rules as your organization’s other shared artifacts.

Beta 期间 Claude Tag 数据没有自动保留期限。Session transcripts、记忆、routines、scopes 和 account links 一直保留，直到本页某个动作删除它们。组织的自定义数据保留设置不适用于 Claude Tag 的 transcripts 或记忆。它适用于 session 发布的 artifacts，后者遵循组织其他共享 artifacts 的同一套保留规则。

Sessions Claude stops using are archived, not deleted. That includes a thread’s session after someone sends @Claude !restart, a session that closed because the thread’s first message was deleted before anyone replied, and the channel-level sessions Claude replaces after about an hour of quiet or a day of age. An archived session keeps its full transcript until the channel’s or workspace’s data is deleted.

Claude 停止使用的 session 被归档，而不是删除。这包括有人发送 @Claude !restart 之后的线程 session、因线程第一条消息在任何人回复之前被删除而关闭的 session，以及 Claude 在约一小时安静或一天年龄之后替换的频道级 session。已归档 session 保留完整 transcript，直到该频道或工作区的数据被删除。

Letting a plan lapse, turning Claude Tag off, or no longer using it doesn’t delete anything on its own. To delete a workspace’s data when you stop using Claude Tag, disconnect the workspace or uninstall the app.

让计划过期、关掉 Claude Tag、或不再使用它，本身都不会删除任何东西。停止使用 Claude Tag 时要删除一个工作区的数据，请断开该工作区或卸载应用。

When an action below deletes data, deletion starts right away and completes in the background. Copies can remain in routine backups for a limited period after that before they age out.

下面某个动作删除数据时，删除立刻开始，在后台完成。之后副本可能在常规备份里再留一段有限时间，然后老化退出。

What each action does to Claude-side data / 每个动作对 Claude 侧数据做什么

The first table covers actions taken in Slack, and the second covers actions taken in Claude. “Claude-side data” means the items listed under What Anthropic stores.

第一张表覆盖在 Slack 里采取的动作，第二张覆盖在 Claude 里采取的动作。「Claude 侧数据」指 Anthropic 存什么 下列出的项目。

Actions in Slack / Slack 里的动作

Action Claude-side data Does Claude keep responding?
A Slack admin uninstalls the Claude app from a workspace Deleted, the same as disconnecting the workspace, plus the app’s installation credential for that workspace. Access bundles, and routines members set up in direct messages, stay No. The app is removed from the workspace
On Enterprise Grid, a Grid admin removes an org-wide installation of the app, from the whole grid or from one of its workspaces Nothing is deleted. To delete the data, an Owner disconnects the grid, or a workspace that has its own pairing, in your Claude admin settings No
A channel is deleted in Slack Deleted: that channel’s sessions and transcripts, including earlier sessions Claude had archived there, its channel memory, and its scope with the routines and artifacts that belong to it. Notes Claude saved to workspace memory from a public channel aren’t tied to the channel and stay until you delete them or disconnect the workspace, as does anything created under the workspace scope rather than the channel’s own Not applicable
A channel is archived in Slack Nothing is deleted. Unarchiving the channel picks its memory, routines, and sessions back up Not while the channel is archived
A public channel is made private Nothing is deleted. Claude archives the channel’s earlier sessions, and new sessions save to the channel’s own memory store. Notes Claude saved to workspace memory while the channel was public stay there, readable from the workspace’s other channels; see Workspace memory Yes
Someone removes Claude from a channel with /remove @Claude Nothing is deleted. The channel’s memory, routines, and past sessions stay on record, and re-adding Claude picks them back up. The channel’s routines keep firing while Claude is out of the channel but can’t post there No
Someone edits a message Claude read Claude receives the edit as a note, and the transcript keeps both the earlier and the edited text Yes
Someone deletes a reply, or deletes a thread’s first message after others have replied Nothing is removed from the transcript, and Claude isn’t notified Yes
Someone deletes a thread’s first message before anyone has replied The session closes and is archived with its transcript, including the deleted message No. The thread is gone
Someone deletes a file they attached in a thread Claude worked in The copy Claude took into the session stays with the session and its transcript Yes
A member’s Slack account is deactivated Nothing is deleted. Their account link, direct-message conversations and notes, and their messages in channel transcripts stay. Channel routines they created keep running, and routines they set up in direct messages keep running until they’re removed from your Claude organization Not to that member
动作 Claude 侧数据 Claude 是否继续应答？
Slack 管理员从工作区卸载 Claude 应用 删除，与断开工作区相同，外加该工作区的应用安装凭证。Access bundles，以及成员在私信里设置的 routines，留下 否。应用已从工作区移除
Enterprise Grid 上，Grid 管理员移除应用的组织级安装（整个 grid 或其中一个工作区） 什么都不删。要删数据，由 Owner 在 Claude 管理设置里断开该 grid，或断开有自己配对的工作区 否
在 Slack 里删除一个频道 删除：该频道的 session 和 transcripts（含 Claude 在那里归档过的更早 session）、其频道记忆，以及属于它的 scope 连同 routines 和 artifacts。Claude 从公开频道存到工作区记忆的笔记不绑在该频道上，会留下直到你删除它们或断开工作区；在工作区 scope 而不是频道自己的 scope 下创建的东西也一样 不适用
在 Slack 里归档一个频道 什么都不删。取消归档会把记忆、routines 和 session 捡回来 频道处于归档时否
公开频道改为私有 什么都不删。Claude 归档该频道更早的 session，新 session 存到频道自己的记忆存储。频道公开时 Claude 存到工作区记忆的笔记留在那里，可从工作区其他频道读取；见 Workspace memory 是
有人用 /remove @Claude 把 Claude 移出频道 什么都不删。频道的记忆、routines 和过去的 session 留在记录上，再加回 Claude 会捡回来。Claude 不在频道时，该频道的 routines 仍会触发，但不能在那里发帖 否
有人编辑 Claude 读过的消息 Claude 把编辑作为一条说明收到，transcript 同时保留更早文本和编辑后文本 是
有人删除一条回复，或在已有他人回复后删除线程第一条消息 transcript 里什么都不移除，Claude 也收不到通知 是
有人在任何人回复之前删除线程第一条消息 session 关闭并连同 transcript 归档，包括被删的那条消息 否。线程没了
有人删除自己在 Claude 工作过的线程里附上的文件 Claude 带进 session 的那份拷贝仍随 session 及其 transcript 是
成员的 Slack 账户被停用 什么都不删。其 account link、私信对话和笔记、以及频道 transcripts 里他们的消息留下。他们创建的频道 routines 继续跑；他们在私信里设置的 routines 继续跑，直到他们被移出 Claude 组织 不对该成员

Actions in Claude / Claude 里的动作

Action Claude-side data Does Claude keep responding?
An Owner disconnects a workspace Deleted: the workspace’s sessions and transcripts, including members’ direct-message conversations there; its channel, workspace, and direct-message memory; the routines set up in its channels and the artifacts published from them; its scopes with their instructions and bundle bindings; and members’ account links. The Slack app and its installation credential stay so a workspace admin can pair again, and Access bundles, routines members set up in direct messages, and what Claude posted in Slack stay too. Pairing the same workspace to the same organization again within a few minutes can cancel the deletion Stops in channels immediately. Direct messages keep working on each member’s own Claude account until the deletion removes that member’s account link
An Owner disconnects an Enterprise Grid The same as disconnecting a workspace, for every workspace in the grid that doesn’t have its own workspace pairing, plus the app’s installation credentials for those workspaces. Workspaces you paired individually stay connected and keep their data until you disconnect them Stops in the affected workspaces
An Owner removes a channel’s scope with Remove this scope in the Claude Tag’s access section Deleted: the channel’s sessions and transcripts recorded up to that moment, including threads still in progress, its memory, its routines, and the artifacts published from it. The Slack channel itself is unchanged Yes. Claude stays in the channel and, when tagged again, starts fresh under the access it inherits from the workspace. To stop it as well, run /remove @Claude or set the scope’s version to Off first
An Owner sets a scope’s version to Off, turns off Claude in Slack for the organization, or turns off direct messages Nothing is deleted. Turning the setting back on resumes with the existing memory, routines, and sessions No, in the affected scope
An Owner detaches a bundle from a scope, or deletes an Access bundle Detaching removes the binding, and deleting a bundle removes its credentials everywhere it was attached. Memory, routines, and transcripts are unaffected Yes, without that access
An Owner deletes entries from a scope’s memory files, or someone in the channel tells Claude to forget an entry The entry is removed from what Claude reads and from the memory files view. Earlier versions of the scope’s memory remain stored with the scope until the scope’s data is deleted Yes
An Owner deletes a routine from the Scheduled work tab, or someone asks Claude to delete it in the channel or direct message where it was set up The routine and the sessions it ran are deleted. Pausing a routine there, or disabling it from the channel, keeps it on record Yes
A member selects Disconnect in the Claude app’s Home tab in Slack Removes the link between their Slack and Claude accounts and revokes the tokens Claude Tag held for them. Their earlier direct-message conversations and notes, and channel work they started, aren’t deleted; those go with the workspace In channels, yes. Direct messages and personal connectors stop working for that member until they reconnect
A member is removed from your Claude organization Nothing is deleted. They lose access within minutes, and routines they set up in direct messages are turned off. Their account link, direct-message conversations, and notes stay until the workspace is disconnected Not to that member
A member deletes their own Claude account On Team and Enterprise plans, deleting an individual account doesn’t remove the Claude Tag data your organization holds about that member, including their direct-message conversations and notes and their account link. A member who wants the link and its tokens removed can select Disconnect in Slack before deleting their account Not to that member
Your Claude organization is deleted All Claude Tag data for every paired workspace is deleted as part of the organization deletion. The Slack app isn’t uninstalled by this, and its installation credential remains until the app is uninstalled, so uninstall it from each workspace in Slack as well No
Your organization adopts Zero Data Retention or another restricted compliance configuration after pairing Nothing already stored is deleted. Disconnect each workspace to delete its data No. Claude stops responding in Slack and new pairings are refused; see Restricted compliance settings block Claude Tag
动作 Claude 侧数据 Claude 是否继续应答？
Owner 断开一个工作区 删除：该工作区的 session 和 transcripts（含成员在那里的私信对话）；其频道、工作区和私信记忆；在其频道里设置的 routines 以及从它们发布的 artifacts；其 scopes 连同指令和 bundle 绑定；以及成员的 account links。Slack 应用及其安装凭证留下，以便工作区管理员可以再配对；Access bundles、成员在私信里设置的 routines，以及 Claude 在 Slack 里发过的内容也留下。几分钟内把同一工作区再配对到同一组织，可以取消这次删除 频道里立刻停止。私信继续在每个成员自己的 Claude 账户上工作，直到删除移除该成员的 account link
Owner 断开一个 Enterprise Grid 与断开工作区相同，作用于 grid 里没有自己工作区配对的每一个工作区，外加那些工作区的应用安装凭证。你单独配对过的工作区保持连接并保留数据，直到你断开它们 在受影响的工作区停止
Owner 在 Claude Tag 的 access 区用 Remove this scope 移除一个频道的 scope 删除：截至那一刻记录的该频道 session 和 transcripts（含仍在进行的线程）、其记忆、其 routines，以及从它发布的 artifacts。Slack 频道本身不变 是。Claude 留在频道里，再次被 Tag 时在从工作区继承的访问下重新开始。若也要停下它，先跑 /remove @Claude 或把该 scope 的版本设为 Off
Owner 把某个 scope 的版本设为 Off、为组织关掉 Claude in Slack，或关掉私信 什么都不删。把设置再打开会带着已有记忆、routines 和 session 恢复 否，在受影响的 scope 里
Owner 从某个 scope 卸下一个 bundle，或删除一个 Access bundle 卸下移除绑定；删除 bundle 会在它被绑过的所有地方移除其凭证。记忆、routines 和 transcripts 不受影响 是，只是没有那份访问
Owner 从某个 scope 的 memory files 删除条目，或频道里有人让 Claude 忘掉一条 该条目从 Claude 所读内容和 memory files 视图里移除。该 scope 记忆的更早版本仍随 scope 存储，直到该 scope 的数据被删除 是
Owner 从 Scheduled work 页删除一条 routine，或有人在设置它的频道或私信里请 Claude 删除它 该 routine 以及它跑过的 session 被删除。在那里暂停，或从频道禁用，会把它留在记录上 是
成员在 Slack 里 Claude 应用的 Home 页选择 Disconnect 移除其 Slack 与 Claude 账户之间的链接，并撤销 Claude Tag 为他们持有的 token。他们更早的私信对话和笔记、以及他们启动的频道工作不会被删；那些跟着工作区走 频道里是。对该成员，私信和个人 connectors 停止工作，直到他们重新连接
成员被移出你的 Claude 组织 什么都不删。他们在几分钟内失去访问，他们在私信里设置的 routines 被关掉。其 account link、私信对话和笔记留下，直到工作区被断开 不对该成员
成员删除自己的 Claude 账户 在 Team 和 Enterprise 计划上，删除个人账户并不会移除组织持有的关于该成员的 Claude Tag 数据，包括其私信对话和笔记及其 account link。想移除链接及其 token 的成员，可以在删账户之前在 Slack 里选择 Disconnect 不对该成员
你的 Claude 组织被删除 每个已配对工作区的全部 Claude Tag 数据作为组织删除的一部分被删除。这不会卸载 Slack 应用，其安装凭证留到应用被卸载，因此也要在 Slack 里从每个工作区卸载它 否
配对之后组织采用 Zero Data Retention 或其他受限合规配置 已经存储的什么都不删。断开每个工作区以删除其数据 否。Claude 停止在 Slack 应答，新配对被拒绝；见 Restricted compliance settings block Claude Tag

Direct messages / 私信

A direct-message conversation with Claude is stored the same way a channel thread is, as a session with a transcript, together with the notes Claude keeps for that conversation. Both are deleted with the workspace the member messaged Claude from, when that workspace is disconnected or the app is uninstalled from it, and when your Claude organization is deleted. They aren’t deleted when the member selects Disconnect in Slack, when the member’s Claude account is deleted on a Team or Enterprise plan, or when an Owner turns off direct messages.

与 Claude 的私信对话按频道线程同样的方式存储：一个带 transcript 的 session，加上 Claude 为该对话保留的笔记。二者随成员向 Claude 发过消息的那个工作区一起删除——当该工作区被断开、应用从它卸载，或你的 Claude 组织被删除时。成员在 Slack 里选择 Disconnect、Team/Enterprise 计划上删除该成员的 Claude 账户、或 Owner 关掉私信时，它们不会被删。

Routines a member set up in a direct message belong to that member’s Claude account. An Owner can delete them from the Scheduled work tab, or the member can ask Claude in the direct message to delete one. They are turned off when the member is removed from your Claude organization, and disconnecting the workspace doesn’t delete them.

成员在私信里设置的 routines 属于该成员的 Claude 账户。Owner 可以从 Scheduled work 页删除它们，或该成员在私信里请 Claude 删除一条。成员被移出 Claude 组织时它们被关掉；断开工作区并不会删除它们。

When a member first opens a direct message with Claude, the welcome Claude writes, which suggests channels based on the member’s recent public-channel activity, runs as a short session that is stored like any other.

成员第一次打开与 Claude 的私信时，Claude 写的欢迎（根据该成员近期公开频道活动建议频道）跑成一个短 session，像其他任何 session 一样被存储。

Delete data or request deletion / 删除数据或请求删除

The controls that delete Claude Tag data, from largest to smallest:

删除 Claude Tag 数据的控件，从大到小：

  • Disconnect a workspace or an Enterprise Grid at claude.ai/admin-settings/claude-tag, or uninstall the app from the workspace in Slack. Deletes all of that workspace’s data. See Revoke a pairing

  • Remove a channel’s scope in the Claude Tag’s access section. Deletes that channel’s data recorded so far

  • Delete a scope’s memory files, from the scope’s options menu, or tell Claude in the channel to forget an entry. See Check and correct what Claude Tag remembers

  • Delete a routine from the Scheduled work tab, or ask Claude to delete it in the channel or direct message where it was set up. See Audit Claude Tag activity

  • 在 claude.ai/admin-settings/claude-tag 断开一个工作区或 Enterprise Grid，或在 Slack 里从工作区卸载应用。删除该工作区的全部数据。见 Revoke a pairing

  • 在 Claude Tag 的 access 区移除一个频道的 scope。删除该频道迄今记录的数据

  • 从 scope 的选项菜单删除其 memory files，或在频道里让 Claude 忘掉一条。见 Check and correct what Claude Tag remembers

  • 从 Scheduled work 页删除一条 routine，或在设置它的频道或私信里请 Claude 删除它。见 Audit Claude Tag activity

There is no control in Slack or in your Claude admin settings that deletes a single thread’s transcript on its own. During the beta, Claude Tag session transcripts and memory aren’t included in your organization’s data exports, and the Compliance API doesn’t list or delete Claude Tag sessions. For a deletion request these controls don’t cover, contact your account team or privacy@anthropic.com.

Slack 或 Claude 管理设置里没有单独删除某一条线程 transcript 的控件。Beta 期间，Claude Tag 的 session transcripts 和记忆不包含在组织的数据导出里，Compliance API 也不列出或删除 Claude Tag session。这些控件覆盖不到的删除请求，联系你的客户团队或 privacy@anthropic.com。

Audit records for deletions / 删除的审计记录

Disconnecting a workspace or an Enterprise Grid, and removing a channel’s scope, are recorded in your organization’s audit log, which you read through the Compliance API. Deletions that start in Slack, such as deleting a channel or uninstalling the app, aren’t recorded there, and neither is a member’s Disconnect in the Home tab.

断开工作区或 Enterprise Grid，以及移除一个频道的 scope，会记入组织的审计日志，通过 Compliance API 阅读。从 Slack 开始的删除（例如删频道或卸载应用）不记在那里，成员在 Home 页的 Disconnect 也不记。

Related resources / 相关资源

1.3.7 - Claude Tag settings map

官方文档 Claude Tag settings map 的中英对照

来源： Claude Tag settings map

Claude Tag settings map / Claude Tag 设置地图

Claude Tag settings map: the admin page for access and behavior, the usage page for spend limits, the in-Slack Configure link for channel instructions, and personal connectors for DMs. Claude Managed Agents is configured separately on the Claude Platform.

Claude Tag 设置地图：管访问与行为的管理页、管 spend limits 的用量页、Slack 里 Configure 链接对应的频道指令，以及私信用的个人 connectors。Claude Managed Agents 在 Claude Platform 上单独配置。

Claude Tag’s settings live on claude.ai, split across a few pages that each own a different kind of setting. Which page you need depends on what you’re changing. The table maps each surface to what it controls.

Claude Tag 的设置在 claude.ai 上，拆到几页，每页管一类设置。你需要哪一页取决于你在改什么。下表把每个面映射到它控制的内容。

Surface Who changes it What it controls
Claude Tag admin page An Owner in your Claude organization Access, behavior, and restrictions for channels, per scope
Usage page An admin Spend limits and each channel’s spend against them
Analytics page Anyone who can view the Analytics dashboard Spend trends, projections, and per-channel reports; read-only
The Configure link in the footer of any Claude reply in a channel Channel members (unless an admin restricts editing) and channel managers for their assigned channels One channel’s instructions and whether Claude replies there without an @-mention. Channel managers also set the channel’s default model, repositories, and connections
Customize > Connectors on your own claude.ai account You Which of your personal tools apply in DMs
界面 谁改 控制什么
Claude Tag 管理页 Claude 组织中的 Owner 按 scope 的频道访问、行为与限制
用量页 管理员 Spend limits，以及各频道相对限额的花费
Analytics 页 能看 Analytics 仪表盘的任何人 花费趋势、预测、按频道报告；只读
频道里任意 Claude 回复页脚的 Configure 链接 频道成员（除非管理员限制编辑）以及被指派频道的 channel managers 一个频道的指令，以及 Claude 是否在没有 @提及时回复。Channel managers 还设定该频道的默认模型、仓库和 connections
你自己 claude.ai 账户上的 Customize > Connectors 你 哪些个人工具在私信里生效

Channel memory and routines aren’t in the table because you change them by talking to Claude in the channel; see what anyone can change from the channel. Owners can review both, as each scope’s memory files and scheduled work, from the Audit page, labeled Activity in the console.

频道记忆和 routines 不在表里，因为你通过在频道里跟 Claude 说话来改它们；见频道里任何人能改什么。Owner 可以从 Audit 页（控制台里标为 Activity）审阅二者，即每个 scope 的 memory files 和 scheduled work。

The Claude Tag admin page / Claude Tag 管理页

Everything an Owner configures for channels lives at claude.ai/admin-settings/claude-tag. Settings there apply per scope (a channel, a workspace, or the whole organization). A scope without its own setting inherits from its parent, and a channel’s setting overrides its workspace’s, so two channels can run with different connections, models, and instructions. Most controls are Owner-only; the permissions table lists each action and who can take it.

Owner 为频道配置的一切都在 claude.ai/admin-settings/claude-tag。那里的设置按 scope 生效（一个频道、一个工作区或整个组织）。没有自己设置的 scope 从父级继承，频道的设置覆盖其工作区的，因此两个频道可以跑不同的 connections、模型和指令。多数控件仅 Owner 可用；权限表列出每项动作以及谁能做。

  • Access bundles: the connections, domain entries, repository grants, and plugins Claude uses in the channels a bundle covers. See Give Claude access.

  • Custom instructions: standing guidance Claude reads in every session on a scope. See Add custom instructions.

  • Default model: the model new sessions in a scope start on. The picker shows the models your organization allows for Claude Code, leaving out any that Claude Tag doesn’t support, so it can be missing models you see in Claude Code itself. See Choose the model for a scope.

  • Auto mode allow rules: plain sentences that pre-approve actions Claude’s permission checker would otherwise flag or stop in a scope’s sessions. See Auto mode allow rules.

  • Workspace pairing and restrictions: which Slack workspaces are paired, whether DMs are allowed, guest-channel behavior, who can invoke Claude, and which generation of the app answers in each scope (on the Team plan, a single Enable Claude Tag switch replaces the per-scope setting). See Restrict where Claude Tag operates.

  • Channel name rules: channel-name patterns that keep Claude out of matching channels or join it automatically to new public ones. See Block or auto-join channels by name.

  • Access bundles： Claude 在 bundle 覆盖的频道里使用的 connections、域名条目、仓库授权和 plugins。见 Give Claude access。

  • Custom instructions： Claude 在某个 scope 的每个 session 里读的常驻指引。见 Add custom instructions。

  • Default model： 某个 scope 里新 session 启动时用的模型。选择器显示组织允许给 Claude Code 用的模型，并去掉 Claude Tag 不支持的，因此可能缺少你在 Claude Code 本身看到的模型。见 Choose the model for a scope。

  • Auto mode allow rules： 用普通句子预先批准 Claude 的权限检查器否则会标记或拦住的动作，作用于某个 scope 的 session。见 Auto mode allow rules。

  • 工作区配对与限制： 哪些 Slack 工作区已配对、是否允许私信、访客频道行为、谁能调用 Claude，以及每个 scope 由哪一代应用应答（Team 计划上，单个 Enable Claude Tag 开关代替按 scope 的设置）。见 Restrict where Claude Tag operates。

  • 频道名规则： 频道名模式，用来让 Claude 远离匹配的频道，或自动加入新的公开频道。见 Block or auto-join channels by name。

Spend limits and usage / Spend limits 与用量

Spend limits live at claude.ai/admin-settings/usage/claude-tag, a different page than the Claude Tag admin page. It holds the organization-wide spend limit, the default spend limit for channels, per-channel limits, and each channel’s spend against its limit. If your organization bills through a reseller, this page is not available. See Set a spend limit for funding the usage balance and what users see when a limit is reached.

Spend limits 在 claude.ai/admin-settings/usage/claude-tag，与 Claude Tag 管理页不同。它容纳全组织 spend limit、频道默认限额、按频道限额，以及各频道相对其限额的花费。若组织通过经销商计费，此页不可用。如何注入用量余额、限额触达时用户看到什么，见 Set a spend limit。

Spend trends live at claude.ai/analytics/claude-tag, the Claude Tag section of the Analytics dashboard. It shows total and projected spend, spend by channel, and spend by kind of work for the period you pick, and anyone with permission to view the Analytics dashboard can open it. It has no controls; see Usage analytics.

花费趋势在 claude.ai/analytics/claude-tag，即 Analytics 仪表盘的 Claude Tag 区。它显示你所选时段的总花费与预测花费、按频道花费、按工作种类花费；有权查看 Analytics 仪表盘的人都能打开。它没有控件；见 Usage analytics。

The Configure page / Configure 页

Every Claude reply in a channel ends with a footer, and its Configure link opens a claude.ai page for that channel; replies in DMs have no Configure link. You can also send @Claude !configure in the channel, and Claude replies with a link to the same page.

频道里每条 Claude 回复都以页脚结束，其中的 Configure 链接打开该频道的 claude.ai 页；私信里的回复没有 Configure 链接。你也可以在频道里发 @Claude !configure，Claude 会回复同一页的链接。

Anyone in the channel who is also a member of your Claude organization can edit the Channel instructions field on that page, unless an admin has restricted editing to admins. The page’s Respond automatically toggle controls whether Claude replies in the channel without an @-mention; see Turn automatic replies on or off.

频道里同时也是 Claude 组织成员的人，都可以编辑该页的 Channel instructions 字段，除非管理员已把编辑限制为仅管理员。该页的 Respond automatically 开关控制 Claude 是否在没有 @提及时回复频道；见 Turn automatic replies on or off。

The page’s Tools and access tab shows the channel’s resolved connections and any allowed domains. Members can see those lists but not change them there. The same tab’s Plugins card lists the channel’s plugins, and members can add plugins there unless an admin has restricted editing to admins. A Routines tab lists the channel’s routines with each one’s schedule, status, and last run.

该页的 Tools and access 页显示频道解析后的 connections 以及任何允许的域名。成员能看见这些列表，但不能在那里改。同一页的 Plugins 卡片列出频道的 plugins，成员可以在那里添加 plugins，除非管理员已把编辑限制为仅管理员。Routines 页列出频道的 routines，含各自的日程、状态和上次运行。

The Configure page and the Custom instructions field on the scope’s panel in admin settings write the same instructions, so a change from either place is visible in the other. See Configure Claude for a channel.

Configure 页与管理设置里该 scope 面板上的 Custom instructions 字段写的是同一套指令，因此任一处的变更在另一处可见。见 Configure Claude for a channel。

On the Enterprise plan, an Owner can name channel managers for a channel. For them, the same page adds editable cards: the channel’s default model on the General tab, and its repositories and access bundles on the Tools and access tab.

Enterprise 计划上，Owner 可以为一个频道指定 channel managers。对他们，同一页会多出可编辑卡片：General 页上的频道默认模型，以及 Tools and access 页上的仓库和 access bundles。

Personal connectors on claude.ai / claude.ai 上的个人 connectors

Connectors you add to your own claude.ai account, under Customize > Connectors, apply only in DMs with Claude, because a DM runs on your own account. A channel uses only the connections an admin attached to it, and personal connectors never apply there. Slack has no connector settings of its own.

你在自己 claude.ai 账户的 Customize > Connectors 下添加的 connectors，只在与 Claude 的私信里生效，因为私信跑在你自己的账户上。频道只用管理员绑给它的 connections，个人 connectors 从不在那里生效。Slack 没有自己的 connector 设置。

See connectors on claude.ai for setting one up, and the troubleshooting entry if a connector you use on claude.ai is missing in Slack.

如何设置见 claude.ai 上的 connectors；若你在 claude.ai 上用的 connector 在 Slack 里缺失，见故障排除条目。

Claude Tag versus Claude Managed Agents / Claude Tag vs. Claude Managed Agents

Claude Managed Agents is a separate product for developers, a pre-built agent harness that runs in managed infrastructure. You configure it on the Claude Platform through the Managed Agents API, and access requires a Claude API key. An agent there is defined by its model, system prompt, tools, MCP servers, and skills. Environments choose where its sessions run (a cloud sandbox, or a self-hosted sandbox on your own infrastructure), and scheduled deployments run it on a cron schedule.

Claude Managed Agents 是面向开发者的另一款产品：跑在托管基础设施上的预构建 agent harness。你在 Claude Platform 上通过 Managed Agents API 配置它，访问需要 Claude API 密钥。那里的 agent 由其模型、系统提示、工具、MCP 服务器和 skills 定义。Environments 选择其 session 跑在哪里（云沙箱，或你们自己基础设施上的自托管沙箱），计划部署按 cron 跑它。

The two products don’t share settings. Nothing on the Claude Tag admin page configures a Managed Agent, and an agent defined on the Claude Platform doesn’t change how Claude behaves in Slack.

两款产品不共享设置。Claude Tag 管理页上没有任何东西配置 Managed Agent；在 Claude Platform 上定义的 agent 也不会改变 Claude 在 Slack 里的行为。

Related resources / 相关资源

1.3.8 - Claude Tag for Claude Code users

官方文档 Claude Tag for Claude Code users 的中英对照

来源： Claude Tag for Claude Code users

Claude Tag for Claude Code users / 给 Claude Code 用户的 Claude Tag

Which parts of a Claude Code setup carry into Claude Tag, which move to admin settings, and how Slack threads map to sessions.

Claude Code 配置里哪些能带到 Claude Tag，哪些改由管理设置接管，以及 Slack 线程如何映射到 session。

Claude Tag runs the same engine as Claude Code. When you tag @Claude in Slack with a task, a session starts in a sandbox that your organization configures, not on your machine. That sandbox is the same infrastructure that runs Claude Code on the web, described in Compute and the sandbox.

Claude Tag 跑的是与 Claude Code 同一套引擎。你在 Slack 里用 @Claude Tag 一项任务时，session 在组织配置的沙箱里开始，而不是在你的机器上。那个沙箱就是网页上 Claude Code 用的同一套基础设施，见 Compute and the sandbox。

If you use Claude Code on the web, a session works the way a web session does, from a fresh clone of your repository rather than from files on your machine. The CLAUDE.md files and skills you checked into that repository apply in the session as they do in a web session.

若你用网页上的 Claude Code，session 的工作方式与 web session 相同：从仓库的全新克隆开始，而不是从你机器上的文件。你提交进该仓库的 CLAUDE.md 文件和 skills 会像在 web session 里一样应用到这个 session。

If you run Claude Code in your terminal, the settings on your own machine don’t reach a session, because the session runs in the sandbox and can’t read your machine. For most of those settings, an admin sets a channel-wide counterpart instead, and a few have no counterpart at all. This page shows what happens when a session starts, which admin settings replace your local ones, and how Slack threads map to sessions.

若你在终端里跑 Claude Code，你机器上的设置到不了 session，因为 session 跑在沙箱里，读不到你的机器。对这些设置中的大多数，管理员会设一套频道级对应物；少数完全没有对应物。本页说明 session 启动时发生什么、哪些管理设置替换你的本地设置，以及 Slack 线程如何映射到 session。

What happens when a session starts / Session 启动时发生什么

A session begins with a fresh sandbox and no repository checked out. Your repository’s Claude Code configuration takes effect only after Claude clones the repository, which happens when your message names a repository that an admin has granted to the channel.

session 从全新沙箱开始，没有检出任何仓库。你仓库里的 Claude Code 配置只有在 Claude 克隆该仓库之后才生效；这发生在你的消息点名了一个管理员已授予该频道的仓库时。

Step What applies
You tag @Claude with a task Your message is the task, and Claude starts work in a sandbox with no repository
Your message names a granted repository Claude clones it into the sandbox
The clone completes CLAUDE.md, .claude/CLAUDE.md, .claude/rules/*.md, and the skills in .claude/skills/ load into the session
步骤 什么生效
你用 @Claude Tag 一项任务 你的消息就是任务，Claude 在没有仓库的沙箱里开始工作
你的消息点名了一个已授予的仓库 Claude 把它克隆进沙箱
克隆完成 CLAUDE.md、.claude/CLAUDE.md、.claude/rules/*.md 以及 .claude/skills/ 里的 skills 加载进 session

Hooks in the repository’s .claude/settings.json don’t run in the session.

仓库 .claude/settings.json 里的 hooks 不会在 session 里运行。

Local settings versus admin settings / 本地设置 vs. 管理设置

A session reads configuration from your repository, not from your machine. The CLAUDE.md files and skills you checked into the repository load when Claude clones it, as described in What happens when a session starts.

session 从你的仓库读配置，而不是从你的机器。你提交进仓库的 CLAUDE.md 文件和 skills 在 Claude 克隆它时加载，见 Session 启动时发生什么。

The settings on your machine never load into a session, because a session runs in the sandbox and can’t read your machine. That includes your ~/.claude directory, your personal settings.json, your shell environment, and the MCP servers you configured locally. They still apply when you run Claude Code in your terminal.

你机器上的设置永远不会加载进 session，因为 session 跑在沙箱里，读不到你的机器。这包括你的 ~/.claude 目录、个人 settings.json、shell 环境，以及你在本地配置的 MCP 服务器。你在终端里跑 Claude Code 时它们仍然生效。

Admin counterparts for local settings / 本地设置的管理侧对应物

The table shows what takes the place of each setting from your machine. Where a counterpart exists, an admin sets it for the whole channel.

下表显示你机器上每项设置由什么替代。有对应物时，由管理员为整个频道设置。

Claude Code setting on your machine In Claude Tag
/model An admin sets the default model per channel, and you can switch models in a thread
Effort level Not configurable. Sessions run at the model’s default effort.
MCP servers in .mcp.json Not loaded, even when .mcp.json is checked into the repository. A session reaches external services only through the connections an admin set for the channel, and each connection holds that service’s credentials.
Secrets and API keys in your environment An admin provisions them as channel connections. The raw key never enters the sandbox. It is added to requests at the network layer.
Environment variables An admin sets them on the environment the channel’s sessions run on, and every session in the channel reads them. There is no per-person environment to customize. The values are readable in every session on the environment, so ask an admin to add secrets as connections instead.
A personal settings.json Not loaded.
A setup script for your workspace An admin sets a setup script on the environment the channel’s sessions run on, and what it installs is in place when each session in the channel starts. For setup that belongs to one repository, use CLAUDE.md install steps instead.
Permission prompts Sessions run in auto mode, where Claude’s permission checker reviews each action and can stop it. An admin pre-approves routine actions with auto mode allow rules instead of you approving in the moment.
你机器上的 Claude Code 设置 在 Claude Tag 里
/model 管理员按频道设默认模型，你可以在线程里切换模型
Effort level 不可配置。Session 以模型的默认 effort 运行。
.mcp.json 里的 MCP 服务器 不加载，即使 .mcp.json 已提交进仓库。session 只通过管理员为该频道设置的 connections 到达外部服务，每条 connection 持有该服务的凭证。
环境里的 secrets 和 API keys 管理员把它们配成频道 connections。原始密钥从不进入沙箱。它在网络层被加到请求上。
环境变量 管理员设在该频道 session 所跑的 environment 上，频道里每个 session 都读它们。没有按人定制的 environment。这些值在该 environment 的每个 session 里可读，因此请管理员把 secrets 加成 connections。
个人 settings.json 不加载。
工作区的 setup script 管理员把 setup script 设在该频道 session 所跑的 environment 上，频道里每个 session 启动时安装内容已就位。属于某一个仓库的安装步骤，改用 CLAUDE.md。
权限提示 Session 跑在 auto mode，Claude 的权限检查器审阅每个动作并可以拦住。管理员用 auto mode allow rules 预先批准常规动作，而不是由你当场批准。

To change what a session can reach, ask an admin to add a connection. The change applies to every session in the channel.

要改变 session 能触及什么，请管理员加一条 connection。变更对该频道里每个 session 生效。

How Slack threads map to sessions / Slack 线程如何映射到 session

You start a session by tagging @Claude in a thread with a task, and that session gets its own sandbox. Each reply in the same thread continues the session, so there is no --continue or /resume to run, and a session stays attached to the thread it started in. See the lifecycle of a request for what happens between replies.

你在线程里用 @Claude Tag 一项任务来开始 session，该 session 有自己的沙箱。同一线程里的每条回复都继续这个 session，因此没有 --continue 或 /resume 可跑，session 一直附着在它开始的那条线程上。回复之间发生什么，见 lifecycle of a request。

Each thread is its own session with its own sandbox, so run parallel tasks in separate threads the way you would in separate terminal tabs. The sandbox is released after a quiet period, but the conversation stays in the thread, and a later reply continues the session. See what survives between replies.

每条线程是自己的 session、自己的沙箱，因此并行任务开在分开的线程里，就像开在分开的终端标签页。安静一段时间后沙箱被释放，但对话留在线程里，稍后的回复会继续这个 session。见 what survives between replies。

Whose credentials a session uses / Session 用谁的凭证

Claude Code acts with your credentials. What a session acts with depends on whether you tag Claude in a channel or in a direct message.

Claude Code 用你的凭证行事。session 用什么，取决于你在频道还是在私信里 Tag Claude。

In a channel / 在频道里

In a channel, Claude acts with credentials of its own, service accounts that an admin provisions. A pull request comes from the Claude GitHub App rather than from you, and a query against a connected service runs with the channel’s credentials no matter who asked. Access is set per channel, not per person.

在频道里，Claude 用它自己的凭证行事——管理员配好的服务账号。拉取请求来自 Claude GitHub App 而不是你；对已连接服务的查询无论谁问，都用频道的凭证。访问按频道设定，不按人。

In a direct message / 在私信里

A direct message runs on your own claude.ai account, with the connectors you added to that account rather than the connections an admin set for the channel, so a DM is the closest match to a Claude Code session on your own credentials.

私信跑在你自己的 claude.ai 账户上，用的是你加到该账户的 connectors，而不是管理员为频道设置的 connections，因此私信是最接近「用你自己凭证的 Claude Code session」的形态。

Steer a session in the thread / 在线程里转向 session

Where you would interrupt Claude Code and edit a file or reprompt, reply in the thread. Corrections and added constraints land as messages, and Claude folds them into the running task.

你会打断 Claude Code 去改文件或重写提示的地方，改成在线程里回复。纠正和追加约束作为消息落地，Claude 把它们折进正在跑的任务。

Keep instructions in channel memory / 把指令放进频道记忆

For instructions that should persist beyond one thread, use channel memory, the instructions Claude keeps for one channel and reads in every session there. Keep repository conventions in CLAUDE.md. Put channel conventions in memory by telling Claude to remember them:

应跨越一条线程持久存在的指令，用频道记忆——Claude 为某一个频道保留、并在那里每个 session 里阅读的指令。仓库惯例放在 CLAUDE.md。频道惯例放进记忆，告诉 Claude 记住它们：

@Claude remember for this channel: reports go out as tables

@Claude remember for this channel: reports go out as tables

See What Claude remembers for how memory is scoped and how to correct it.

记忆如何划定范围、如何纠正，见 What Claude remembers。

Claude Tag versus a bot you build on the API / Claude Tag vs. 你用 API 自建的机器人

A Slack bot you build on the Claude API is software your team writes and hosts. It calls the API with your key, holds its own Slack tokens, and has the tools and memory you code into it.

你基于 Claude API 自建的 Slack 机器人，是团队自己写、自己托管的软件。它用你的密钥调 API，持有自己的 Slack token，工具和记忆是你编进去的。

Claude Tag is Anthropic’s hosted Slack app. It takes care of the parts you would otherwise build.

Claude Tag 是 Anthropic 托管的 Slack 应用。它包办了你否则得自己建的那些部分。

  • Hosting. Each thread gets a Claude Code session in a sandbox Anthropic runs, or in the environment your organization pins, under an agent identity of its own.

  • Credentials. An admin gives Claude connections to your tools, and Agent Proxy attaches the credentials at the network boundary, outside the sandbox.

  • Customization. Custom instructions, channel memory, and routines are built in.

  • Governance and billing. Access controls and spend limits are set in claude.ai admin settings, and usage is billed to the organization’s usage balance.

  • 托管。 每条线程在 Anthropic 跑的沙箱里（或组织钉住的 environment 里）获得一个 Claude Code session，落在它自己的 agent identity 下。

  • 凭证。 管理员给 Claude 配到你们工具的 connections，Agent Proxy 在网络边界、沙箱之外附上凭证。

  • 定制。 自定义指令、频道记忆和 routines 是内建的。

  • 治理与计费。 访问控制和 spend limits 在 claude.ai 管理设置里设定，用量记到组织的用量余额。

Related resources / 相关资源

2 - [论文]重新思考面向智能体服务系统的 AI 云基础设施

[论文]Rethinking AI Cloud Infrastructure for Agentic Serving Systems

2.1 - 论文介绍

论文信息和介绍

基本信息

发表时间： 2026年7月31日

论文PDF： https://arxiv.org/pdf/2607.23933

论文 HTML 页面： https://arxiv.org/html/2607.23933v2

作者信息

2.2 - 论文英文版本

论文英文版本

内容来源： https://arxiv.org/html/2607.29069v1

Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework

Abstract

Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimentation framework that separates task semantics from execution configurations, reconstructs cross-component agent trajectories with correlated system telemetry, and exposes stateful tool execution through a consistent interface across heterogeneous sandbox substrates. We use Aries to conduct reproducible experiments on open agent harnesses and benchmarks. We complement these experiments with production traces from a commercial platform, grounding low-level systems research in observed production behavior. Our results show that (1) token-centric metrics miss non-inference bottlenecks, (2) retaining additional context yields diminishing accuracy benefits while reducing serving capacity, and (3) tool sandboxes alternate between long idle periods and short resource bursts, while current snapshot-based state management makes aggressive suspension costly. A complementary security analysis further highlights the need to reduce the sandbox attack surface. We then discuss the vision for agent-native serving systems designed around trajectory-level metrics, adaptive context management, elastic sandbox resource management, and sandboxes with minimized attack surface.

1 Introduction

Large Language Models (LLMs) are rapidly evolving from prompt responders into autonomous agents. Open-source agents such as OpenClaw and Hermes have collectively attracted over half a million GitHub stars within months of their public releases (OpenClaw Contributors, 2026; Nous Research, 2026), while closed-source commercial agents such as Claude and Codex are used by millions of people (CryptoBriefing, 2026). An agent places the model inside a feedback loop: it observes the current state, selects an action, executes that action through a tool, and uses the result to decide how the task should continue (Luo et al., 2025; Luo et al., 2026; Yao et al., 2022). This shifts the system problem from serving isolated model requests to sustaining end-to-end agent trajectories. Agent serving therefore no longer ends when the model emits tokens; each model response is only one step in an execution loop whose progress depends on both generated text and runtime state.

Agent Serving Goes Beyond Inference. Like conventional LLM serving, agentic workloads are often hosted in cloud AI infrastructure (Chen et al., 2025; Luo et al., 2026), but they impose fundamentally different system requirements. Conventional LLM systems optimize individual model calls: scheduling and resource management are focused on producing the next model response efficiently (Kwon et al., 2023; Zheng et al., 2024; Yu et al., 2022; Zhong et al., 2024). This abstraction works when the surrounding application logic remains outside the serving system. In agent serving, however, each model call is only one step in a task loop. Faster GPU decoding remains useful, but end-to-end progress also depends on retained context, tool execution, and sandbox management. Agent serving therefore requires a holistic, full-stack system design, beyond just fast inference serving.

Task Progress Needs System Visibility. Existing evaluation methods capture only part of agent execution. Agent benchmarks measure whether a task reaches the correct outcome (Padigela et al., 2025; Zhou et al., 2024; Merrill et al., 2026; Liu et al., 2024), whereas system benchmarks typically measure isolated model calls, stateless functions, or task transactions (Reddi et al., 2020; Ustiugov et al., 2021; Ferdman et al., 2012). Neither of the above connects task progress and correctness to cross-layer runtime behavior, while task–execution entanglement and substrate-dependent tool behavior hinder controlled comparison.

We use the agent trajectory as the unit of observation. A trajectory is the ordered record of model invocations, tool interactions, harness decisions, and the final task outcome. Based on this abstraction, we introduce Aries , a modular experimentation framework that separates task semantics from execution configuration, reconstructs cross-component trajectories with correlated telemetry, and standardizes tool execution across sandbox substrates, enabling reproducible comparisons and stage-level attribution of stalls and failures. We use Aries to conduct reproducible experiments on open benchmarks and complement them with production traces from a commercial platform (anonymized), thereby grounding low-level systems research in observed production behavior.1

Our experiments reveal limitations that the conventional model-call metrics hide. Harness and tool execution can account for a heavy part of the end-to-end latency (e.g., up to 48%), so LLM-engine token throughput may diverge from task-level execution efficiency. Context retention creates an accuracy–capacity trade-off: controlled sweeps show that avoiding context overflows can raise task success from 55% to 95%, but gains plateau at workload-dependent thresholds, while production traces and controlled telemetry show that retained state reduces serving capacity. Production and controlled measurements also reveal mostly idle-but-bursty tool sandboxes, for which snapshot-based suspension remains economically infeasible. Our security analysis further motivates sandbox designs with minimal attack surfaces. Based on these insights, we discuss the vision for agent-native serving systems that feature trajectory-aware observability, control planes for trajectory context management and for the elasticity of agent-aware sandboxes, and sandbox designs that minimize the attack surface of tool execution.

2 Background and Motivation

Figure 1.Overview of conventional LLM-serving and emerging agent-serving systems.

We first discuss the differences between the conventional LLM serving and emerging agentic workloads, and their implications for modern cloud AI infrastructure. We then discuss the existing benchmarking frameworks and why they fall short for systems research in agentic systems.

2.1 From Model Requests to Agent Trajectories

Conventional LLM serving systems are organized around the execution of individual model requests. As shown in Fig. 1a, the serving runtime processes a prompt, generates output tokens, manages accelerator memory, and returns a response (Zhong et al., 2024; Yu et al., 2022). Scheduling and memory management are therefore scoped primarily to the active invocation. Depending on the cache policy, the engine may retain, offload, reuse, or reclaim the invocation’s KV-cache state after execution (Kwon et al., 2023; Pan et al., 2025; Zheng et al., 2024; Qin et al., 2025). This request-centric design naturally prioritizes metrics such as time to first token, time per output token, request latency, and aggregate token throughput (Kwon et al., 2023; Zhong et al., 2024; Patel et al., 2024; Hu et al., 2024b; Yu et al., 2022; Zheng et al., 2024).

Agent serving departs from this invocation-centric abstraction. As shown in Fig. 1b, the harness repeatedly invokes the LLM, dispatches tool calls to a sandboxed runtime, incorporates tool results into later prompts, and decides when the task is completed (Luo et al., 2026; Liu et al., 2024; Yao et al., 2022). A single user request may therefore span many dependent model and tool steps.

In this paper, we define an agent trajectory as the ordered sequence of LLM invocations, tool calls, tool results, and harness decisions produced while serving one user request. Individual model requests are steps within this larger execution unit rather than independent transactions. Trajectory state persists across these steps. The harness may represent this state as message history or structured events, while the serving layer may materialize reusable portions as cached model state, such as KV-cache prefixes. At the same time, model-generated actions execute on the host side within an isolated sandbox. Agent serving consequently spans GPU inference, host-side orchestration, persistent context state, and sandboxed execution, with dependencies across all four components.

2.2 Limitations of Existing Benchmarking Frameworks

Existing benchmarks answer complementary questions at incompatible granularities. Capability-oriented benchmarks evaluate whether an agent completes a realistic task, but largely treat the execution path as a black box (Zhou et al., 2024; Gur et al., 2024; Jimenez et al., 2024; Merrill et al., 2026; Mialon et al., 2024; Xie et al., 2024; Yao et al., 2024). Systems benchmarks can measure end-to-end performance and resource behavior, but typically consider individual requests or transactions as their unit of analysis (Reddi et al., 2020; Ferdman et al., 2012; Gan et al., 2019; Ustiugov et al., 2021).

None of the prior frameworks and benchmarks reveal how system behavior affects progress within a multi-step agent task. Task semantics are often entangled with harness- and runtime-specific choices, execution events remain fragmented across model, harness, and sandbox components, and stateful tool behavior depends on the underlying substrate. Agent serving therefore creates a measurement gap between semantic task progress and cross-layer system execution, making it difficult to isolate configuration effects, reconstruct complete trajectories, and attribute resource bottlenecks to task-level delays and failures.

3 Aries: Trajectory-Level Telemetry

g) Figure 2.Aries architecture. The blue-colored components are instrumentation in the agent-serving loop, dedicated to bridging the measurement gap.

The goal of Aries is to bridge the measurement gap between semantic task outcomes and cross-layer system behavior by treating an agent trajectory as the first-class citizen. This entails three challenges. First, task semantics are often entangled with execution choices, hindering controlled comparison across stacks. Second, agent execution spans independently managed components, making trajectories difficult to reconstruct or attribute from local logs. Third, tool execution is both stateful and substrate-dependent, complicating consistent comparison of its behavior and resource cost. Aries addresses these challenges with separate task and execution specifications, a shared trajectory schema that correlates cross-layer telemetry, and a substrate-independent interface for stateful tool execution and instrumentation.

Fig. 2 presents the architecture of Aries. Task and execution loaders validate the specifications; the runner coordinates the harness and accesses LLM backends and sandbox substrates through adapters; and the telemetry collector merges framework events with cross-layer telemetry into a unified record. This design separates task semantics from execution configuration, preserves cross-component provenance, and standardizes stateful tool execution across sandbox substrates.

Preserving task semantics across configurations. Agent frameworks often embed task semantics within harness-specific logic, so replacing a runtime component may inadvertently change the task being evaluated. Aries therefore separates the task specification, which captures the workload and its success semantics, from the execution specification, which selects the harness, model backend, sandbox, and telemetry stack. Dedicated loaders validate the two specifications, while LLM backend and sandbox adapters bind them to a concrete execution stack without task-specific changes. This separation enables controlled comparisons across configurations.

Reconstructing cross-component trajectories. Agent runs may contain overlapping, failed, or retried operations across independently managed components, making component-local logs and timestamps insufficient for reliable reconstruction and attribution. Aries propagates trajectory and event identifiers across model and tool boundaries and records causal and ordering metadata under a common event schema. The telemetry collector then associates component metrics with these events, revealing where delays and resource pressure arise along the trajectory.

Making stateful tool execution observable and portable. Tool calls operate on a persistent environment whose state affects subsequent trajectory steps, while different sandbox substrates expose different lifecycle and telemetry interfaces. Aries standardizes environment setup, tool invocation, state continuity, result collection, and telemetry hooks, while sandbox adapters map these operations onto concrete substrates. This makes tool execution a first-class, consistently observable stage of the trajectory while supporting different sandbox technologies.

4 Why Is Modern Cloud AI Infrastructure a Poor Fit for Agents?

Using Aries, we evaluate whether request-centric modern cloud AI infrastructure is sufficient for agent serving. Specifically, we ask which components determine the end-to-end critical path, how long-lived trajectory context affects serving efficiency and accuracy, how resources are utilized across LLM backends and tool sandboxes, and the adequacy of existing tool-isolation mechanisms. We first analyze an eight-hour workday production trace, including request lengths from ten LLM-serving instances and one-minute CPU and memory samples from 100 agent sandboxes, to establish high-level workload behavior at deployment scale. As the operational traces do not expose synchronized cross-layer events, we then reproduce the observed patterns with open harnesses and datasets and use Aries for deeper system-level analysis.

For this controlled analysis, Aries integrates OpenHands (Wang et al., 2026), Hermes Agent (Nous Research, 2026), and OpenClaw (OpenClaw Contributors, 2026), and evaluates serving tasks from SWE-Bench Pro (Deng et al., 2025), Terminal-Bench 2 (Team, 2025), and DeepResearch Bench (Du et al., 2025). We sample 20 tasks from each benchmark to achieve 50% task success rate (important for the accuracy studies) with OpenClaw and repeat each task five times. Unless otherwise stated, experiments use Qwen3.6-35B-A3B-FP8 (Qwen Team, 2026), served locally by SGLang (Zheng et al., 2024) on a 96-core node with one NVIDIA H100 GPU with 94 GB of HBM.

4.1 Harnesses & Tools Matter as Much as Models

Existing LLM-serving engines optimize token-centric metrics such as TTFT and TPOT (Zhong et al., 2024; Kwon et al., 2023; Zheng et al., 2024). These metrics omit the host-side intervals during which the harness processes model outputs or waits for tool execution, even though the agent cannot advance until these steps complete.

Figure 3.End-to-end average latency breakdown. Harness and tool execution contribute latency on par with LLM calls.

Figure 4. Latency distributions for LLM inference vs. tool execution. Tool calls exhibit a severe long tail spanning multiple orders of magnitude.

As the per-step timing information is too sensitive to collect in the commercial cluster, we base the analysis of the utility of existing metrics on the setup with public datasets and harnesses which record these events under a shared trajectory identifier and time base.

Our profiling highlights two key system insights: (1) Critical-Path Shift to Harness and Tool Execution: As shown in Fig. 3, tool execution is an important latency contributor, contributing from 13% to 48% of the total latency across datasets and harnesses, while reaching 48% for Hermes Agent on Terminal-Bench 2. Beyond tool execution, the harness spends up to 9% of end-to-end time on orchestration and other host-side processing, further extending the interval between consecutive LLM calls and potentially prolonging the residency of trajectory state in GPU memory. (2) Long-Tailed Latency Profiles: Fig. 4 shows that while LLM latency remains important, tool calls exhibit a severe long tail that spans multiple orders of magnitude. Together, these results show that these massive host-side delays frequently dominate execution time, meaning that solely optimizing accelerator token generation cannot resolve the actual agent bottlenecks.

Takeaway 1: End-to-end task duration depends heavily on harness and tool sandbox execution, rather than model inference alone.

4.2 Long Context Dictates Efficiency & Accuracy

In conventional LLM serving, context lifetimes are typically scoped to the request-response cycle, allowing engines to reclaim KV cache pages post-query unless explicitly retained or offloaded for reuse (Kwon et al., 2023; Zheng et al., 2024; Zhong et al., 2024; Hu et al., 2024a; Qin et al., 2025; Liu et al., 2025). Conversely, long-horizon agents generate monotonically growing histories that persist across the entire trajectory. While existing frameworks apply semantic compression to fit system limits (Rasmussen et al., 2025; Xu et al., 2026; Chhikara et al., 2025), unmanaged context expansion still locks up physical GPU token pools and reduces serving concurrency (Li et al., 2025; Kariyappa and Suh, 2026). Retaining excessive history can increase tail delay, while aggressive compression may discard information required by later steps and reduce task accuracy (Kang et al., 2025; Kariyappa and Suh, 2026). Resolving this accuracy-efficiency tension requires shifting from passive memory allocation to trajectory-aware state management that balances the resource cost and future utility of retained context.

We first compare the context lengths observed in agentic serving and conventional inference, both retrieved from the production platform. Then, we characterize the trade-offs associated with the context budget and how they affect the accuracy and efficiency of agentic serving with the public harnesses and datasets. In the latter analysis, across all evaluated benchmarks, we sweep the context budget for both OpenClaw and Hermes Agent (omitting OpenHands for brevity) to determine when additional retained history stops providing meaningful accuracy gains. We then quantify the cost of context retention when serving 32 concurrent active agentic sessions with a 256K context size, sampling backend resource utilization once per second.

Figure 5. Context length and resident capacity comparison across conversation and agent requests in the production cluster. Agent workloads consume larger per-request KV footprints than conventional serving, sharply reducing the number of processed requests in parallel by an LLM backend.

Figure 6.Compaction and context overflow counts with different context window sizes, and the corresponding task success rate. Larger windows ease overflow/compaction pressure, but task success plateaus past workload-specific thresholds.

We make three key observations: (1) Trajectory State Amplification: Production traces reveal substantially larger agent contexts than those observed in conventional inference workloads, which matches our results with the public datasets and harnesses. As quantified in Fig. 5, the average agent context consumes a substantially larger KV footprint than conventional inference. Under a fixed 25GB budget with the Qwen3.6 model, this state amplification reduces the maximum resident capacity from over 176 concurrent serving requests to merely 40—a 4.4× reduction. (2) Context Sufficiency Thresholds: Using the open datasets and harnesses, we show that long-context retention improves accuracy only up to a task-specific threshold, beyond which additional history, albeit consuming more memory, provides little to no accuracy benefits. As shown in Fig. 6, OpenClaw on DeepResearch Bench reaches peak success (95%) at 160K context, coinciding with the elimination of context overflows. Conversely, Terminal-Bench 2 and SWE-Bench Pro plateau or degrade once overflow pressure is removed, performing best at small or mid-size windows rather than the maximum context size. (3) Capacity-induced Batch Limitation: our results with public datasets further show that large retained state severely constrains serving capacity. Fig. and Fig. show that memory pressure limits effective batching despite a 32-session backlog; token-pool utilization peaks at 97%, while the engine sustains a median of only 22 concurrent requests. Under this pressure, the scheduler preempts active sessions, and the p95 internal queueing delay reaches 7.9s (Fig. ).

Figure 7. CDFs over the one-hour window of the agent serving. The workload sustains high concurrency and KV pressure, while long waits are concentrated in the request tail.

Takeaway 2: Retaining context beyond its task-dependent utility yields diminishing accuracy benefits while increasing KV-cache occupancy, limiting batching, and prolonging queueing delays. The system must therefore actively manage trajectory state to balance task accuracy against serving efficiency.

4.3 Tool Sandboxes Must Be Elastic

Agent serving introduces CPU-based processing to the critical path: it alternates between idle intervals during LLM inference and short CPU/memory bursts during tool execution. Static peak resource provisioning avoids throttling these tool calls but leaves resources stranded a significant portion of the time. Cloud systems such as AWS Lambda MicroVMs (Amazon Web Services, 2026a) and Google Agent Sandbox (Google Cloud, 2026) attempt to reduce waste by applying a stateful scale-to-zero approach via sandbox snapshot capture and restore (C/R). However, agentic tool execution patterns lead to a high number of capture-restore cycles, incurring additional costs, as we show below.

Figure 8. CDFs of Harness CPU and memory utilization aggregating measurements taken every minute from the production cluster. The values are normalized for trace anonymization purposes.

We collect CPU and memory utilization from over 100 sandboxes on the same commercial platform, each co-hosting an OpenClaw-style harness and its tool calls. To confirm the trend and for in-depth analysis, we also collect CPU and memory utilization of harness and tool sandboxes in a controlled environment every second and analyze their distributions for OpenClaw and Hermes using the cloud DeepSeek V4 Flash API, which is production-grade and highly stable for tool call generation accuracy. We then analytically replay the recorded tool-call traces against the AWS Lambda MicroVMs pricing model (Amazon Web Services, 2026b) under different sandbox keepalive policies as in serverless deployments (Shahrad et al., 2020; Fuerst and Sharma, 2021). We report the instance cost (CPU and memory usage), the snapshot C/R cost, and their sum. The ∞ keepalive point corresponds to the persistent sandbox. DeepResearch is excluded from this comparison due to the tools’ low CPU usage.

Figure 9.Tool and harness CPU and memory utilization CDFs aggregating measurements taken every second. The tool environment is mostly idle, with high CPU utilization spikes during tool calls.

Figure 10.Absolute customer-visible cost of snapshot-based stateful scale-to-zero under different keepalive timeouts based on AWS Lambda MicroVM billing model. Short keepalive reduces instance cost but triggers frequent snapshot C/R, while longer keepalive reduces C/R by keeping the sandbox resident for more idle time.

Fig. 8 shows that CPU compute and memory utilization in the production setting is spiky. We reproduce this with the public datasets and harnesses (Fig. 9), showing that agent tool sandboxes spend most of their trajectories at low utilization, yet exhibit sharp spikes in CPU and memory demand during tool execution. Across harnesses and workloads, CPU and memory remain idle for more than 80% and 50% of the time, respectively. At the same time, the tool-call-conditioned distributions show much more intensive resource use: the portion of time with idle resources drops below 20% for both CPU and memory. Fig. 10 shows that both users and cloud providers have an incentive to reduce keepalive to lower their costs: users avoid paying for idle sandboxes, while providers reclaim stranded capacity sooner. Yet current snapshot-based mechanisms and pricing prevent this alignment: if tool sandboxes are torn down after each tool call, their cost falls by 65%, but repeated snapshot C/R raises the total customer cost to 4.9× compared to the persistent baseline. Keeping tool sandboxes for longer reduces C/R overhead but increases idle time. Thus, today’s system forces a false choice between paying for idle capacity and paying for repeated snapshots and restores. However, these results suggest that the total cost can be reduced by 3× in an ideal system compared to the persistent-sandbox baseline.

Takeaway 3: Agent sandboxes are mostly idle but bursty when tools execute. Both users and providers should benefit from idle resource reclamation, yet current state-management mechanisms and pricing make aggressive suspension uneconomical.

4.4 Tool Sandboxes Are Under Continuous Siege

Cloud infrastructure is already designed to handle untrusted code execution, and uses the same mechanisms to sandbox agentic workloads. Beyond LLM-specific challenges, e.g., KV caches side-channels (Chu et al., 2025) or sensitive input data (Luo et al., 2025), agents have the ability to autonomously discover and exploit vulnerabilities (Ullah et al., 2025; Zhang et al., 2026; Xu et al., 2025). Frontier models already found advanced vulnerabilities in production system software, such as OS kernels and VMMs (Carlini et al., 2026), hinting at the possibility of AI agents breaching cloud isolation.

We use the share of Linux kernel CVEs identified by AI tools as a proxy for the ability of AI agents to find vulnerabilities autonomously. We compute that share as the proportion of commits in the Linux CVEs that explicitly mention using AI tools to find and fix vulnerabilities. We display our findings in Fig. 11.

Figure 11.Total and relative share of Linux kernel CVEs reported as found by AI agents or with AI assistance. There is a sharp increase in AI-found CVEs in Q2 of 2026.

AI tools are playing an increasing role in finding vulnerabilities. On the Linux kernel, this is especially true since the introduction of Sashiko, an agentic patch review system (Gushchin, 2026). The trend holds across CVE publishers, with a 3.5× increase in published CVEs for the second quarter of 2026 compared to previous records (Emberson, 2026). Exploiting known CVEs is also becoming cheap: agentic systems can develop exploits for a few dollars (Ullah et al., 2025). Agentic workloads are already highly capable of carrying sophisticated attacks against the cloud infrastructure. Therefore, sandboxes should expose a minimal attack surface, but also rely on defense-in-depth mechanisms to anticipate latent vulnerabilities.

Takeaway 4: AI agents are highly capable of autonomously finding and exploiting vulnerabilities. Tool sandboxes should be designed to withstand advanced attacks and to anticipate increasing offensive model capabilities by reducing the attack surface.

5 Future Research Directions

Figure 12.Agent-native AI serving system vision.

Our results suggest that agent-serving systems should coordinate optimization decisions across the trajectory rather than manage model inference, context state, and tool execution independently. As illustrated in Fig. 12, this requires jointly optimizing task progress, capacity, and resources through four co-designed directions across the Runtime, Control, and Execution layers: (1) trajectory-capturing metrics spanning all layers; (2) control plane for trajectory-aware context management; (3) elastic tool execution management via sandbox control; and (4) minimizing the tool sandbox attack surface.

5.1 Trajectory-capturing Metrics

Future agent-serving metrics should continuously quantify how system execution contributes to task progress. As illustrated in Fig. 13, evaluation should connect end-to-end outcomes, such as task success and completion rate, with low-level measures of execution speed and cost, including latency, generated tokens, and resource consumption. Task goodput and solved tasks per generated token provide useful endpoint metrics, but they become available only after a trajectory completes and cannot guide runtime decisions.

A key research direction is therefore to derive online utility signals from trajectory events. Such signals should estimate the incremental progress contributed by each model call, harness decision, and tool interaction while accounting for its time and resource cost. They should distinguish productive actions from repeated or non-advancing steps and aggregate into end-to-end measures of task accuracy, execution speed, and system cost. This connection would allow low-level system optimizations to be evaluated by their actual contribution to agent task completion.

Figure 13.Token throughput, solved tasks per 1 million generated tokens, and solved tasks per hour. Token throughput can diverge from task goodput and execution efficiency in agent serving.

5.2 Control Plane for Trajectory-Aware Context Management

Our measurements in §4.2 reveal a workload-dependent trade-off between context sufficiency and serving capacity. Additional context increases KV-cache pressure and queueing delay, even when it provides only a small improvement in task success. This motivates a trajectory-aware control plane that monitors task and memory states, determines how context should be retained and placed, and directs the LLM backend to perform context compaction, KV-cache compression, or state transfer. (1) Adaptive Context Retention. The control plane should estimate the future utility of context as a trajectory evolves and request compaction or compression when the expected accuracy benefit of retained state no longer justifies its memory cost. Such decisions must remain conservative enough to preserve information needed by later model invocations. (2) Hierarchical State Placement. The control plane should coordinate long-lived context placement across accelerator and lower-cost memory tiers rather than leaving all state resident on the accelerator. It should trigger KV-cache migration when the released accelerator capacity outweighs the transfer and restoration overhead.

5.3 Control Plane for Agent-Aware Elasticity

Our characterization in §4.3 shows that host-side resource demand is highly intermittent. Static provisioning wastes capacity, while conventional, horizontal autoscaling mechanisms are too expensive at such granularity. Future agent-serving platforms therefore require an elasticity control plane designed around model-driven, stateful execution. (1) Vertical, Model-Aware Compute Management. Research should explore how signals from the model, harness, and runtime can guide timely vertical scaling of individual sandboxes to satisfy the identified volatile demand for memory and core count fluctuations. Key questions include which signals reliably predict demand, how uncertainty should be handled, and how resources should be allocated across competing trajectories. (2) State Management for Elastic Execution. Because sandboxes retain files, processes, caches, and network state, elasticity requires decoupling execution state from physical placement and resource allocation. Research is needed on what state must remain resident, what can be externalized or reconstructed, and how to preserve consistency while resizing, suspending, or relocating sandboxes.

5.4 Sandboxes with a Minimal Attack Surface

AI agents are increasingly capable of autonomously finding and exploiting vulnerabilities. Thus, cloud infrastructure should anticipate increasingly sophisticated adversarial workloads. (1) Defense-in-Depth and Minimizing Attack Surface. Sandboxes should be tailored to the workload to reduce the attack surface by restricting tool, network, and filesystem access based on the target tasks. Tool execution should leverage both host-side VM-level isolation and guest-side kernel-level sandboxing. For instance, workloads such as deep research do not need direct bash access, but can be granted a sandboxed JavaScript runtime within the guest. Also, the system design has to minimize the attack surface exposed to side-channel timing attacks across the agentic stack. (2) Formal Security Guarantees. In the long term, formal verification of security properties is the only path to certifying the absence of vulnerabilities. Migration to verified systems, such as seL4 (Klein et al., 2009), as a host hypervisor or guest kernel, will require major but necessary system overhaul to ensure the cloud remains secure despite models’ increasing offensive capabilities.

6 Conclusion

Agent serving turns conventional LLM serving into long-lived, stateful trajectories spanning model execution, harness logic, and sandboxes. We present Aries, a telemetry framework that combines reproducible experiments with production traces to characterize task and system behavior. Our findings motivate trajectory-capturing metrics, control planes for context management and agent-aware elasticity, and sandboxes with minimal attack surfaces.

2.3 - 论文中文翻译版本

论文中文翻译版本

以 Aries 实验框架重新思考面向智能体服务系统的 AI 云基础设施

摘要

自主智能体(Agent)将重复推理与持久化上下文、沙箱化工具执行相结合,对传统 LLM 服务构成了新的挑战。我们提出 Aries——一个全栈实验框架,它可将任务语义与执行配置分离、通过关联的系统遥测重建跨组件智能体轨迹,并在异构沙箱基底上以一致的接口暴露有状态工具执行。我们使用 Aries 在开源智能体平台与基准测试上开展可复现实验,并用来自商业平台的线上生产数据轨迹加以补充,从而将底层系统研究建立在观测到的真实生产行为之上。研究结果表明:(1) 以 Token 为中心的指标会遗漏非推理环节的瓶颈;(2) 保留额外上下文带来的精度收益逐渐递减,同时会降低服务容量;(3) 工具沙箱在长时间空闲与短时资源突发之间交替,而当前基于快照的状态管理使得激进式挂起成本高昂。配套的安全分析进一步凸显了缩小沙箱攻击面的必要性。最后,我们讨论了对智能体原生(Agent-Native)服务系统的愿景,其设计围绕轨迹级指标、自适应上下文管理、弹性沙箱资源管理,以及攻击面最小化的沙箱展开。

1 引言

大语言模型(LLM)正迅速从提示词应答器演进为自主智能体。开源智能体如 OpenClaw 和 Hermes,在公开发布后的数月内便累计吸引了超过五十万个 GitHub star(OpenClaw Contributors, 2026; Nous Research, 2026);而 Claude、Codex 等闭源商业智能体已被数百万用户使用(CryptoBriefing, 2026)。智能体将模型置于反馈回路之中:它观测当前状态、选择动作、通过工具执行该动作,并利用执行结果决定任务应如何继续(Luo et al., 2025; Luo et al., 2026; Yao et al., 2022)。这使系统问题从"服务孤立的模型请求"转变为"维持端到端的智能体轨迹"。因此,智能体服务不再止步于模型生成 Token 的时刻;每一次模型响应只是执行循环中的一步,其推进既取决于生成的文本,也取决于运行时状态。

智能体服务已超越推理范畴。 与传统 LLM 服务类似,智能体化工作负载通常托管于云端 AI 基础设施之中(Chen et al., 2025; Luo et al., 2026),但它们对系统提出了根本不同的需求。传统 LLM 系统优化的是单次模型调用:调度与资源管理聚焦于高效地产生下一个模型响应(Kwon et al., 2023; Zheng et al., 2024; Yu et al., 2022; Zhong et al., 2024)。当外围应用逻辑位于服务系统之外时,这一抽象是成立的。然而在智能体服务中,每一次模型调用只是任务循环中的一步。更快的 GPU 解码依然有用,但端到端进度还取决于上下文的保留、工具的执行以及沙箱的管理。因此,智能体服务需要的不仅是快速的推理服务,更是整体的、全栈式的系统设计。

任务进度需要系统层面的可见性。 现有的评估方法只捕捉到智能体执行的一部分。智能体基准测试衡量的是任务是否达到正确结果(Padigela et al., 2025; Zhou et al., 2024; Merrill et al., 2026; Liu et al., 2024),而系统基准测试通常衡量的是孤立的模型调用、无状态函数或任务事务(Reddi et al., 2020; Ustiugov et al., 2021; Ferdman et al., 2012)。这两类方法都没有将任务进度与正确性同跨层运行时行为联系起来,同时任务与执行的纠缠以及依赖底层基质的工具行为,也阻碍了受控对比。

我们采用 智能体轨迹(agent trajectory) 作为观测单元。轨迹是对模型调用、工具交互、harness 决策以及最终任务结果的按序记录。基于这一抽象,我们提出 Aries——一个模块化实验框架,它将任务语义与执行配置分离,通过关联遥测重建跨组件轨迹,并在不同沙箱基底上标准化工具执行,从而支持可复现的对比以及停滞与失败的分阶段归因。我们使用 Aries 在开源基准上开展可复现实验,并以来自商业平台(匿名化)的生产轨迹加以补充,从而将底层系统研究建立在观测到的真实生产行为之上。

我们的实验揭示了传统"模型调用指标"所掩盖的局限。harness 与工具执行可占端到端延迟的很大比重(例如高达 48%),因此 LLM 引擎的 Token 吞吐可能与任务级执行效率相背离。上下文保留会带来精度—容量的权衡:受控扫描显示,避免上下文溢出可将任务成功率从 55% 提升至 95%,但增益在依赖工作负载的阈值处趋于平缓;而生产轨迹与受控遥测表明,保留的状态会降低服务容量。生产与受控测量还揭示出大部分"空闲但突发"的工具沙箱,对这类沙箱而言,基于快照的挂起在经济上仍不可行。我们的安全分析进一步推动了攻击面最小化沙箱设计的必要性。基于这些洞察,我们讨论了对智能体原生服务系统的愿景,其特征包括:轨迹感知的可观测性、面向轨迹上下文管理与智能体感知沙箱弹性的控制面,以及使工具执行攻击面最小化的沙箱设计。

2 背景与动机

图 1. 传统 LLM 服务与新兴智能体服务系统的概览。

我们首先讨论传统 LLM 服务与新兴智能体化工作负载之间的差异,及其对现代云 AI 基础设施的启示。随后讨论现有基准框架的不足,以及它们为何难以胜任智能体系统的系统研究。

2.1 从模型请求到智能体轨迹

传统 LLM 服务系统围绕单个模型请求的执行进行组织。如图 1a](https://arxiv.org/html/2607.29069v1#S2.F1)a) 所示,服务运行时处理提示词、生成输出 Token、管理加速器内存并返回响应(Zhong et al., 2024; Yu et al., 2022)。因此,调度与内存管理主要限定于当前激活的调用范围。根据缓存策略,引擎可以在执行后保留、卸载、复用或回收该调用的 KV 缓存状态(Kwon et al., 2023; Pan et al., 2025; Zheng et al., 2024; Qin et al., 2025)。这种以请求为中心的设计自然优先关注首 Token 延迟、每输出 Token 时间、请求延迟与聚合 Token 吞吐等指标(Kwon et al., 2023; Zhong et al., 2024; Patel et al., 2024; Hu et al., 2024b; Yu et al., 2022; Zheng et al., 2024)。 智能体服务则偏离了这种以调用为中心的抽象。如图 1b](https://arxiv.org/html/2607.29069v1#S2.F1)b) 所示,harness 反复调用 LLM、将工具调用分派到沙箱化运行时、将工具结果纳入后续提示词,并决定任务何时完成(Luo et al., 2026; Liu et al., 2024; Yao et al., 2022)。因此,单个用户请求可能跨越许多相互依赖的模型步骤与工具步骤。

本文中,我们将 智能体轨迹(agent trajectory) 定义为服务单个用户请求过程中产生的 LLM 调用、工具调用、工具结果与 harness 决策的有序序列。单个模型请求只是这一更大执行单元中的步骤,而非独立事务。轨迹状态在这些步骤之间持续存在。harness 可能将该状态表示为消息历史或结构化事件,而服务层可能将可复用的部分物化为缓存的模型状态(如 KV 缓存前缀)。与此同时,模型生成的动作在主机侧于隔离沙箱内执行。因此,智能体服务横跨 GPU 推理、主机侧编排、持久化上下文状态与沙箱化执行四大组件,且这四个组件之间存在相互依赖。

2.2 现有基准框架的局限

现有基准以互不兼容的粒度回答互补的问题。面向能力的基准评估智能体能否完成现实任务,但大多将执行路径视为黑盒(Zhou et al., 2024; Gur et al., 2024; Jimenez et al., 2024; Merrill et al., 2026; Mialon et al., 2024; Xie et al., 2024; Yao et al., 2024)。系统基准能够测量端到端性能与资源行为,但通常以单个请求或事务作为分析单元(Reddi et al., 2020; Ferdman et al., 2012; Gan et al., 2019; Ustiugov et al., 2021)。

以往的框架与基准都无法揭示系统行为如何影响多步骤智能体任务中的进度。任务语义往往与 harness 及运行时特定的选择纠缠在一起,执行事件在模型、harness 与沙箱各组件间呈碎片化分布,而有状态工具行为又依赖于底层基质。因此,智能体服务在"语义任务进度"与"跨层系统执行"之间形成了测量鸿沟,使得难以隔离配置影响、重建完整轨迹,以及将资源瓶颈归因于任务级延迟与失败。

3 Aries:轨迹级遥测

图 2. Aries 架构。蓝色组件是智能体服务循环中的插桩(Instrumentation),专门用于弥合测量鸿沟。

Aries 的目标是通过将智能体轨迹视为一等公民,弥合"语义任务结果"与"跨层系统行为"之间的测量鸿沟。这带来三项挑战。其一,任务语义常与执行选择纠缠,阻碍跨技术栈的受控对比。其二,智能体执行横跨独立管理的组件,使得仅凭本地日志难以重建或归因轨迹。其三,工具执行既有状态又依赖底层基质,使其行为与资源成本的对比难以一致。Aries 通过"任务与执行规格分离"“关联跨层遥测的共享轨迹模式"以及"面向有状态工具执行与插桩的、与底层基质无关的接口"来解决这些挑战。

图 2 展示了 Aries 的架构。任务加载器与执行加载器负责校验规格;运行器协调 harness,并通过适配器访问 LLM 后端与沙箱基底;遥测收集器将框架事件与跨层遥测合并为统一记录。这一设计将任务语义与执行配置分离,保留跨组件来源信息,并跨沙箱基底标准化有状态工具执行。

跨配置保留任务语义。 智能体框架往往将任务语义嵌入 harness 特有的逻辑中,因此替换运行时组件可能无意中改变被评估的任务。为此,Aries 将任务规格(捕获工作负载及其成功语义)与执行规格(选择 harness、模型后端、沙箱与遥测技术栈)分离。专用加载器对两种规格进行校验,而 LLM 后端与沙箱适配器将其绑定到具体执行栈,无需针对任务的修改。这一分离支持跨配置的受控对比。

重建跨组件轨迹。 智能体运行可能包含跨独立管理组件的重叠、失败或重试操作,使得组件本地日志与时间戳不足以进行可靠的重建与归因。Aries 在模型与工具边界传播轨迹与事件标识符,并在统一事件模式下记录因果与排序元数据。随后遥测收集器将组件指标关联到这些事件,从而揭示延迟与资源压力在轨迹中的产生位置。

使有状态工具执行可观测且可移植。 工具调用作用于一个持久化环境,其状态会影响后续轨迹步骤,而不同的沙箱基底暴露不同的生命周期与遥测接口。Aries 对环境设置、工具调用、状态连续性、结果收集与遥测挂钩进行标准化,同时沙箱适配器将这些操作映射到具体基底。这使得工具执行成为轨迹中一等公民的、一致可观测的阶段,同时支持不同的沙箱技术。

4 为何现代云 AI 基础设施不适合智能体?

利用 Aries,我们评估以请求为中心的现代云 AI 基础设施是否足以支撑智能体服务。具体而言,我们研究:哪些组件决定端到端关键路径;长生命周期轨迹上下文如何影响服务效率与精度;LLM 后端与工具沙箱之间的资源利用情况;以及现有工具隔离机制的充分性。我们首先分析一个八小时工作日的生产轨迹,包括来自十个 LLM 服务实例的请求长度,以及来自 100 个智能体沙箱的每分钟 CPU 与内存采样,以在部署规模上建立高层工作负载行为。由于运维轨迹未暴露同步的跨层事件,我们随后使用开源 harness 与数据集复现观测到的模式,并借助 Aries 开展更深入的系统级分析。

为进行这一受控分析,Aries 集成了 OpenHands(Wang et al., 2026)、Hermes Agent(Nous Research, 2026)与 OpenClaw(OpenClaw Contributors, 2026),并评估来自 SWE-Bench Pro(Deng et al., 2025)、Terminal-Bench 2(Team, 2025)与 DeepResearch Bench(Du et al., 2025)的服务任务。我们从每个基准各采样 20 个任务,使 OpenClaw 达到 50% 的任务成功率(这对精度研究很重要),并每个任务重复五次。除非另有说明,实验使用由 SGLang(Zheng et al., 2024)在本地提供的 Qwen3.6-35B-A3B-FP8(Qwen Team, 2026),运行于一个 96 核节点上,配备一块 94 GB HBM 的 NVIDIA H100 GPU。

4.1 Harness 和工具与模型同等重要

现有 LLM 服务引擎优化的是以 Token 为中心的指标,如 TTFT 与 TPOT(Zhong et al., 2024; Kwon et al., 2023; Zheng et al., 2024)。这些指标忽略了 harness 处理模型输出或等待工具执行的主机侧时间间隔,尽管在完成这些步骤之前智能体无法继续推进。

图 3. 端到端平均延迟分解。harness 和工具执行带来的延迟与 LLM 调用相当。

图 4. LLM 推理与工具执行的延迟分布。工具调用呈现出跨越多个数量级的严重长尾。

由于逐步计时信息在商业集群中过于敏感而难以收集,我们基于公开数据集与 harness(它们在共享的轨迹标识符与时间基准下记录这些事件)来分析现有指标的效用。

我们的剖析揭示了两个关键的系统洞察:

  1. 关键路径向 harness 与工具执行转移:如图 3 所示,工具执行是重要的延迟贡献者,在不同数据集与 harness 下贡献了总延迟的 13% 至 48%,其中 Hermes Agent 在 Terminal-Bench 2 上达到 48%。除工具执行外,harness 还花费高达 9% 的端到端时间用于编排及其他主机侧处理,进一步拉长了连续 LLM 调用之间的间隔,并可能延长轨迹状态在 GPU 内存中的驻留时间。

  2. 长尾延迟分布: 图 4 表明,虽然 LLM 延迟依然重要,但工具调用呈现出跨越多个数量级的严重长尾。综合来看,这些结果说明,大量主机侧延迟常常主导执行时间,这意味着仅优化加速器的 Token 生成无法解决真实的智能体瓶颈。

结论 1: 端到端任务时长在很大程度上取决于 harness 与工具沙箱的执行,而不仅仅是模型推理。

4.2 长上下文决定效率与精度

在传统 LLM 服务中,上下文生命周期通常限定于请求—响应循环,使引擎能在查询结束后回收 KV 缓存页,除非显式保留或卸载以复用(Kwon et al., 2023; Zheng et al., 2024; Zhong et al., 2024; Hu et al., 2024a; Qin et al., 2025; Liu et al., 2025)。

相反,长时程智能体生成单调增长的历史记录,并贯穿整个轨迹持续存在。尽管现有框架采用语义压缩以适配系统限制(Rasmussen et al., 2025; Xu et al., 2026; Chhikara et al., 2025),但不受管理的上下文扩张仍会锁定物理 GPU Token 池,并降低服务并发度(Li et al., 2025; Kariyappa and Suh, 2026)。保留过多历史可能增加尾延迟,而激进压缩可能丢弃后续步骤所需的信息,从而降低任务精度(Kang et al., 2025; Kariyappa and Suh, 2026)。化解这一精度—效率张力,需要从被动的内存分配转向轨迹感知的状态管理,在保留上下文的资源成本与其未来效用之间取得平衡。

我们首先对比生产平台上智能体服务与传统推理中观测到的上下文长度。随后,我们刻画与上下文预算相关的权衡,以及它们如何借助公开 harness 与数据集影响智能体服务的精度与效率。在后一分析中,我们针对所有被评估的基准,对 OpenClaw 与 Hermes Agent 的上下文预算进行扫描(为简洁起见省略 OpenHands),以确定保留额外历史在何时不再带来有意义的精度提升。随后,我们在 256K 上下文大小下服务 32 个并发活跃智能体会话,以每秒一次的频率采样后端资源利用率,量化上下文保留的成本。

图 5. 生产集群中对话请求与智能体请求的上下文长度与驻留容量对比。智能体工作负载的每请求 KV 占用比传统服务更大,大幅降低了 LLM 后端可并行处理的请求数。

图 6. 不同上下文窗口大小下的压缩与上下文溢出次数,及对应的任务成功率。更大的窗口缓解了溢出/压缩压力,但任务成功率在工作负载特定阈值后趋于平缓。

我们得到三点关键观察:

  1. 轨迹状态放大: 生产轨迹显示,智能体上下文显著大于传统推理工作负载中观测到的上下文,这与我们在公开数据集与 harness 上的结果一致。如图 5 所量化,平均智能体上下文消耗的 KV 占用显著大于传统推理。在 Qwen3.6 模型、固定 25GB 预算下,这种状态放大使最大驻留容量从 176 个并发服务请求骤降至仅 40 个——即 4.4× 的缩减。

  2. 上下文充分性阈值: 借助开源数据集与 harness,我们证明长上下文保留仅在达到任务特定阈值前提升精度,超过该阈值后,额外历史尽管消耗更多内存,却几乎不再带来精度收益。如图 6 所示,OpenClaw 在 DeepResearch Bench 上于 160K 上下文处达到峰值成功率(95%),此时恰好消除了上下文溢出。相反,Terminal-Bench 2 与 SWE-Bench Pro 在溢出压力消除后趋于平缓甚至下降,其最佳性能出现在较小或中等窗口,而非最大上下文。

  3. 容量导致的批处理受限: 我们在公开数据集上的结果进一步表明,大规模保留状态严重制约服务容量。图中数据表明,尽管存在 32 个会话的积压,内存压力仍限制了有效批处理;Token 池利用率峰值达到 97%,而引擎仅维持中位数 22 个并发请求。在此压力下,调度器抢占活跃会话,p95 内部排队延迟达到 7.9 秒。

图 7. 智能体服务一小时窗口内的 CDF(累积分布函数)。工作负载维持高并发与高 KV 压力,而长时间等待集中在请求尾部。

结论 2: 保留超出任务依赖效用的上下文,只会带来递减的精度收益,同时增加 KV 缓存占用、限制批处理并延长排队延迟。因此,系统必须主动管理轨迹状态,以在任务精度与服务效率之间取得平衡。

4.3 工具沙箱必须具有弹性

智能体服务将基于 CPU 的处理引入关键路径:它在 LLM 推理期间的空闲间隔与工具执行期间短暂的 CPU/内存突发之间交替。静态的峰值资源供给虽能避免限制这些工具调用,却会在相当长的时间内造成资源闲置。AWS Lambda MicroVMs(Amazon Web Services, 2026a)与 Google Agent Sandbox(Google Cloud, 2026)等云系统试图通过"沙箱快照捕获与恢复(C/R)“实现 有状态缩至零(scale-to-zero) 来减少浪费。然而,智能体化工具执行模式会导致大量捕获—恢复周期,带来额外成本,我们将在下文中展示。

图 8. 生产集群中每分钟聚合测量的 harness CPU 与内存利用率 CDF。数值已归一化以进行轨迹匿名化。

我们收集同一商业平台上超过 100 个沙箱(每个沙箱共托管一个 OpenClaw 风格 harness 及其工具调用)的 CPU 与内存利用率。为确认趋势并进行深入分析,我们还在受控环境中以每秒一次的频率收集 harness 与工具沙箱的 CPU 和内存利用率,并使用云端 DeepSeek V4 Flash API(生产级、对工具调用生成精度高度稳定)分析 OpenClaw 与 Hermes 的分布。随后,我们参照无服务器部署中的不同沙箱保活策略(Shahrad et al., 2020; Fuerst and Sharma, 2021),针对 AWS Lambda MicroVMs 计费模型(Amazon Web Services, 2026b)对记录的工具调用轨迹进行解析式重放。我们报告实例成本(CPU 与内存使用)、快照 C/R 成本及其总和。保活时间为无穷大的点对应持久沙箱。由于 DeepResearch 工具 CPU 占用较低,本对比将其排除。

图 9. 每秒聚合测量的工具与 harness CPU 与内存利用率 CDF。

工具环境大部分时间处于空闲,在工具调用期间出现高 CPU 利用率尖峰。

图 10. 基于 AWS Lambda MicroVM 计费模型,不同保活超时下基于快照的有状态缩至零的绝对客户可见成本。较短的保活时间降低实例成本,但触发频繁的快照 C/R;较长的保活时间通过让沙箱在更多空闲时间内保持驻留来减少 C/R。

图 8 显示,生产环境中的 CPU 计算与内存利用率呈尖峰状。我们使用公开数据集与 harness 复现了这一现象(图 9),表明智能体工具沙箱在轨迹的大部分时间里利用率较低,却在工具执行期间呈现出](https://arxiv.org/html/2607.29069v1#S4.F9)),%E8%A1%A8%E6%98%8E%E6%99%BA%E8%83%BD%E4%BD%93%E5%B7%A5%E5%85%B7%E6%B2%99%E7%AE%B1%E5%9C%A8%E8%BD%A8%E8%BF%B9%E7%9A%84%E5%A4%A7%E9%83%A8%E5%88%86%E6%97%B6%E9%97%B4%E9%87%8C%E5%88%A9%E7%94%A8%E7%8E%87%E8%BE%83%E4%BD%8E,%E5%8D%B4%E5%9C%A8%E5%B7%A5%E5%85%B7%E6%89%A7%E8%A1%8C%E6%9C%9F%E9%97%B4%E5%91%88%E7%8E%B0%E5%87%BA) CPU 与内存需求的急剧尖峰。在各类 harness 与工作负载中,CPU 与内存在超过 80% 与 50% 的时间里处于空闲。与此同时,以工具调用为条件的分布显示出更密集的资源使用:CPU 与内存的空闲时间占比均降至 20% 以下。图 10 表明,用户与云提供商都有动力缩短保活时间以降低成本:用户避免为空闲沙箱付费,而提供商能更早回收闲置容量。然而,当前的基于快照的机制与定价阻碍了这种一致性:如果在每次工具调用后拆除工具沙箱,其成本下降 65%,但反复的快照 C/R 使客户总成本相比持久基线升至 4.9 倍。将工具沙箱保留更久可降低 C/R 开销,但会增加空闲时间。因此,当今系统迫使人们在"为闲置容量付费"与"为反复快照与恢复付费"之间做出错误取舍。不过,这些结果表明,在理想系统中,总成本相比持久沙箱基线可降低 3 倍。

结论 3: 智能体沙箱大部分时间处于空闲,但在工具执行时呈突发性。用户与提供商都应从闲置资源回收中获益,然而当前的状态管理机制与定价使激进式挂起不具经济性。

4.4 工具沙箱正面临持续围攻

云基础设施本已为处理不可信代码执行而设计,并使用同样的机制对智能体化工作负载进行沙箱化。除 LLM 特有的挑战(如 KV 缓存侧信道(Chu et al., 2025)或敏感输入数据(Luo et al., 2025))外,智能体还具备自主发现并利用漏洞的能力(Ullah et al., 2025; Zhang et al., 2026; Xu et al., 2025)。前沿模型已能在操作系统内核、VMM 等生产系统软件中发现高级漏洞(Carlini et al., 2026),这预示着 AI 智能体存在突破云隔离的可能性。 我们以"AI 工具发现的 Linux 内核 CVE 占比"作为衡量 AI 智能体自主发现漏洞能力的代理指标。我们将该占比定义为 Linux CVE 中明确提及使用 AI 工具发现并修复漏洞的提交所占的比例。我们的发现展示于图 11。](https://arxiv.org/html/2607.29069v1#S4.F11)%E3%80%82)

图 11. 报告由 AI 智能体或借助 AI 辅助发现的 Linux 内核 CVE 的总数与相对占比。2026 年第二季度,AI 发现的 CVE 急剧增加。

AI 工具在发现漏洞中扮演着越来越重要的角色。在 Linux 内核上,自引入智能体化补丁审查系统 Sashiko 以来尤其如此(Gushchin, 2026)。这一趋势在各大 CVE 发布方中均成立,2026 年第二季度已发布的 CVE 相比此前的纪录增长了 3.5 倍(Emberson, 2026)。利用已知 CVE 也日益廉价:智能体系统仅需几美元即可开发出利用代码(Ullah et al., 2025)。智能体化工作负载已具备对云基础设施发起复杂攻击的高度能力。因此,沙箱应暴露最小的攻击面,同时还应依赖纵深防御机制以应对潜在漏洞。

结论 4: AI 智能体具备高度的自主发现与利用漏洞的能力。工具沙箱的设计应能抵御高级攻击,并通过缩小攻击面来预期不断增强的进攻性模型能力。

5 未来研究方向

图 12. 智能体原生 AI 服务系统愿景。

我们的结果表明,智能体服务系统应跨轨迹协调优化决策,而不是独立地管理模型推理、上下文状态与工具执行。如图 12 所示,这需要通过四个协同设计方向,在运行时(Runtime)、控制面(Control)与执行(Execution)三个层面联合优化任务进度、容量与资源:(1) 覆盖所有层的轨迹捕获指标;(2) 面向轨迹感知上下文管理的控制面;(3) 通过沙箱控制实现弹性工具执行管理;(4) 最小化工具沙箱攻击面。

5.1 轨迹捕获指标

未来的智能体服务指标应持续量化系统执行对任务进度的贡献。如图 13 所示,评估应将任务成功率、完成率等端到端结果,与延迟、生成 Token 数、资源消耗等执行速度与成本的底层度量相连接。任务有效吞吐与"每生成 Token 解决的已解决问题数"提供了有用的端点指标,但它们仅在轨迹完成后才可用,无法指导运行时决策。 因此,一个关键研究方向是从轨迹事件中推导在线效用信号。这类信号应估计每次模型调用、harness 决策与工具交互所带来的增量进度,同时计入其时间与资源成本。它们应区分有效动作与重复或非推进性步骤,并聚合为任务精度、执行速度与系统成本的端到端度量。这种连接将使得底层系统优化能够依据其对智能体任务完成的实际贡献进行评估。

图 13. Token 吞吐、每 100 万生成 Token 解决的已解决问题数,以及每小时解决的已解决问题数。在智能体服务中,Token 吞吐可能与任务有效吞吐及执行效率相背离。

5.2 面向轨迹感知上下文管理的控制面

我们在第 4.2 节的测量揭示了"上下文充分性"与"服务容量"之间随工作负载变化的权衡。额外上下文会增加 KV 缓存压力与排队延迟,即便其对任务成功率的提升微乎其微。这推动了轨迹感知控制面的诞生:它监控任务与内存状态,决定上下文应如何保留与放置,并指导 LLM 后端执行上下文压缩、KV 缓存压缩或状态迁移。(1) 自适应上下文保留。 控制面应随轨迹演进估计上下文的未来效用,并在保留状态的预期精度收益不再抵偿其内存成本时请求压缩或压缩处理。此类决策必须足够保守,以保留后续模型调用所需的信息。(2) 分层状态放置。 控制面应协调长生命周期上下文在加速器与低成本内存层级之间的放置,而非让所有状态驻留在加速器上。当释放的加速器容量超过迁移与恢复开销时,它应触发 KV 缓存迁移。

5.3 面向智能体感知弹性的控制面

我们在第 4.3 节的刻画表明,主机侧资源需求高度间歇。静态供给浪费容量,而传统水平自动扩缩机制在此粒度下过于昂贵。因此,未来的智能体服务平台需要围绕"模型驱动的有状态执行"设计的弹性控制面。(1) 垂直的、模型感知的计算管理。 研究应探索如何利用来自模型、harness 与运行时的信号,及时指导单个沙箱的垂直扩缩,以满足已识别的内存与核数波动的波动性需求。关键问题包括:哪些信号能可靠预测需求、如何处理不确定性,以及如何在相互竞争的轨迹之间分配资源。(2) 面向弹性执行的状态管理。 由于沙箱保留文件、进程、缓存与网络状态,弹性要求将执行状态与物理放置及资源分配解耦。需要研究:哪些状态必须保持驻留、哪些可以外部化或重建,以及在调整大小、挂起或迁移沙箱时如何保持一致性。

5.4 攻击面最小的沙箱

AI 智能体正日益具备自主发现与利用漏洞的能力。因此,云基础设施应预期日益复杂的对抗性工作负载。(1) 纵深防御与最小化攻击面。 沙箱应针对工作负载量身定制,通过根据目标任务限制工具、网络与文件系统访问来缩小攻击面。工具执行应同时利用主机侧 VM 级隔离与客户机侧内核级沙箱化。例如,深度研究等工作负载无需直接 bash 访问,可授予其客户机内沙箱化的 JavaScript 运行时。此外,系统设计还必须最小化智能体化技术栈中暴露给侧信道时序攻击的攻击面。(2) 形式化安全保障。 长远来看,对安全属性的形式化验证是认证"不存在漏洞"的唯一途径。向 seL4(Klein et al., 2009)等已验证系统迁移(作为主机虚拟机监控程序或客户机内核),将需要重大但必要的系统重构,以确保在模型进攻能力不断增强的情况下云依然安全。

6 结论

智能体服务将传统 LLM 服务转变为横跨模型执行、harness 逻辑与沙箱的长期有状态轨迹。我们提出 Aries——一个将可复现实验与生产轨迹相结合以刻画任务与系统行为的遥测框架。我们的发现推动了轨迹捕获指标、面向上下文管理与智能体感知弹性的控制面,以及攻击面最小化沙箱的发展。

2.4 - 论文双语对照版本

论文中英文双语对照版本

Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework

以 Aries 实验框架重新思考面向智能体服务系统的 AI 云基础设施

Abstract / 摘要

Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimentation framework that separates task semantics from execution configurations, reconstructs cross-component agent trajectories with correlated system telemetry, and exposes stateful tool execution through a consistent interface across heterogeneous sandbox substrates. We use Aries to conduct reproducible experiments on open agent harnesses and benchmarks. We complement these experiments with production traces from a commercial platform, grounding low-level systems research in observed production behavior. Our results show that (1) token-centric metrics miss non-inference bottlenecks, (2) retaining additional context yields diminishing accuracy benefits while reducing serving capacity, and (3) tool sandboxes alternate between long idle periods and short resource bursts, while current snapshot-based state management makes aggressive suspension costly. A complementary security analysis further highlights the need to reduce the sandbox attack surface. We then discuss the vision for agent-native serving systems designed around trajectory-level metrics, adaptive context management, elastic sandbox resource management, and sandboxes with minimized attack surface.

自主智能体(Agent)将重复推理与持久化上下文、沙箱化工具执行相结合,对传统 LLM 服务构成了新的挑战。我们提出 Aries——一个全栈实验框架,它可将任务语义与执行配置分离、通过关联的系统遥测重建跨组件智能体轨迹,并在异构沙箱基底上以一致的接口暴露有状态工具执行。我们使用 Aries 在开源智能体平台与基准测试上开展可复现实验,并用来自商业平台的线上生产数据轨迹加以补充,从而将底层系统研究建立在观测到的真实生产行为之上。研究结果表明:(1) 以 Token 为中心的指标会遗漏非推理环节的瓶颈;(2) 保留额外上下文带来的精度收益逐渐递减,同时会降低服务容量;(3) 工具沙箱在长时间空闲与短时资源突发之间交替,而当前基于快照的状态管理使得激进式挂起成本高昂。配套的安全分析进一步凸显了缩小沙箱攻击面的必要性。最后,我们讨论了对智能体原生(Agent-Native)服务系统的愿景,其设计围绕轨迹级指标、自适应上下文管理、弹性沙箱资源管理,以及攻击面最小化的沙箱展开。

1 Introduction / 引言

Large Language Models (LLMs) are rapidly evolving from prompt responders into autonomous agents. Open-source agents such as OpenClaw and Hermes have collectively attracted over half a million GitHub stars within months of their public releases (OpenClaw Contributors, 2026; Nous Research, 2026), while closed-source commercial agents such as Claude and Codex are used by millions of people (CryptoBriefing, 2026). An agent places the model inside a feedback loop: it observes the current state, selects an action, executes that action through a tool, and uses the result to decide how the task should continue (Luo et al., 2025; Luo et al., 2026; Yao et al., 2022). This shifts the system problem from serving isolated model requests to sustaining end-to-end agent trajectories. Agent serving therefore no longer ends when the model emits tokens; each model response is only one step in an execution loop whose progress depends on both generated text and runtime state.

大语言模型(LLM)正迅速从提示词应答器演进为自主智能体。开源智能体如 OpenClaw 和 Hermes,在公开发布后的数月内便累计吸引了超过五十万个 GitHub star(OpenClaw Contributors, 2026; Nous Research, 2026);而 Claude、Codex 等闭源商业智能体已被数百万用户使用(CryptoBriefing, 2026)。智能体将模型置于反馈回路之中:它观测当前状态、选择动作、通过工具执行该动作,并利用执行结果决定任务应如何继续(Luo et al., 2025; Luo et al., 2026; Yao et al., 2022)。这使系统问题从"服务孤立的模型请求"转变为"维持端到端的智能体轨迹"。因此,智能体服务不再止步于模型生成 Token 的时刻;每一次模型响应只是执行循环中的一步,其推进既取决于生成的文本,也取决于运行时状态。

Agent Serving Goes Beyond Inference. Like conventional LLM serving, agentic workloads are often hosted in cloud AI infrastructure (Chen et al., 2025; Luo et al., 2026), but they impose fundamentally different system requirements. Conventional LLM systems optimize individual model calls: scheduling and resource management are focused on producing the next model response efficiently (Kwon et al., 2023; Zheng et al., 2024; Yu et al., 2022; Zhong et al., 2024). This abstraction works when the surrounding application logic remains outside the serving system. In agent serving, however, each model call is only one step in a task loop. Faster GPU decoding remains useful, but end-to-end progress also depends on retained context, tool execution, and sandbox management. Agent serving therefore requires a holistic, full-stack system design, beyond just fast inference serving.

智能体服务已超越推理范畴。 与传统 LLM 服务类似,智能体化工作负载通常托管于云端 AI 基础设施之中(Chen et al., 2025; Luo et al., 2026),但它们对系统提出了根本不同的需求。传统 LLM 系统优化的是单次模型调用:调度与资源管理聚焦于高效地产生下一个模型响应(Kwon et al., 2023; Zheng et al., 2024; Yu et al., 2022; Zhong et al., 2024)。当外围应用逻辑位于服务系统之外时,这一抽象是成立的。然而在智能体服务中,每一次模型调用只是任务循环中的一步。更快的 GPU 解码依然有用,但端到端进度还取决于上下文的保留、工具的执行以及沙箱的管理。因此,智能体服务需要的不仅是快速的推理服务,更是整体的、全栈式的系统设计。

Task Progress Needs System Visibility. Existing evaluation methods capture only part of agent execution. Agent benchmarks measure whether a task reaches the correct outcome (Padigela et al., 2025; Zhou et al., 2024; Merrill et al., 2026; Liu et al., 2024), whereas system benchmarks typically measure isolated model calls, stateless functions, or task transactions (Reddi et al., 2020; Ustiugov et al., 2021; Ferdman et al., 2012). Neither of the above connects task progress and correctness to cross-layer runtime behavior, while task–execution entanglement and substrate-dependent tool behavior hinder controlled comparison.

任务进度需要系统层面的可见性。 现有的评估方法只捕捉到智能体执行的一部分。智能体基准测试衡量的是任务是否达到正确结果(Padigela et al., 2025; Zhou et al., 2024; Merrill et al., 2026; Liu et al., 2024),而系统基准测试通常衡量的是孤立的模型调用、无状态函数或任务事务(Reddi et al., 2020; Ustiugov et al., 2021; Ferdman et al., 2012)。这两类方法都没有将任务进度与正确性同跨层运行时行为联系起来,同时任务与执行的纠缠以及依赖底层基质的工具行为,也阻碍了受控对比。

We use the agent trajectory as the unit of observation. A trajectory is the ordered record of model invocations, tool interactions, harness decisions, and the final task outcome. Based on this abstraction, we introduce Aries , a modular experimentation framework that separates task semantics from execution configuration, reconstructs cross-component trajectories with correlated telemetry, and standardizes tool execution across sandbox substrates, enabling reproducible comparisons and stage-level attribution of stalls and failures. We use Aries to conduct reproducible experiments on open benchmarks and complement them with production traces from a commercial platform (anonymized), thereby grounding low-level systems research in observed production behavior.1

我们采用 智能体轨迹(agent trajectory) 作为观测单元。轨迹是对模型调用、工具交互、harness 决策以及最终任务结果的按序记录。基于这一抽象,我们提出 Aries——一个模块化实验框架,它将任务语义与执行配置分离,通过关联遥测重建跨组件轨迹,并在不同沙箱基底上标准化工具执行,从而支持可复现的对比以及停滞与失败的分阶段归因。我们使用 Aries 在开源基准上开展可复现实验,并以来自商业平台(匿名化)的生产轨迹加以补充,从而将底层系统研究建立在观测到的真实生产行为之上。

Our experiments reveal limitations that the conventional model-call metrics hide. Harness and tool execution can account for a heavy part of the end-to-end latency (e.g., up to 48%), so LLM-engine token throughput may diverge from task-level execution efficiency. Context retention creates an accuracy–capacity trade-off: controlled sweeps show that avoiding context overflows can raise task success from 55% to 95%, but gains plateau at workload-dependent thresholds, while production traces and controlled telemetry show that retained state reduces serving capacity. Production and controlled measurements also reveal mostly idle-but-bursty tool sandboxes, for which snapshot-based suspension remains economically infeasible. Our security analysis further motivates sandbox designs with minimal attack surfaces. Based on these insights, we discuss the vision for agent-native serving systems that feature trajectory-aware observability, control planes for trajectory context management and for the elasticity of agent-aware sandboxes, and sandbox designs that minimize the attack surface of tool execution.

我们的实验揭示了传统"模型调用指标"所掩盖的局限。harness 与工具执行可占端到端延迟的很大比重(例如高达 48%),因此 LLM 引擎的 Token 吞吐可能与任务级执行效率相背离。上下文保留会带来精度—容量的权衡:受控扫描显示,避免上下文溢出可将任务成功率从 55% 提升至 95%,但增益在依赖工作负载的阈值处趋于平缓;而生产轨迹与受控遥测表明,保留的状态会降低服务容量。生产与受控测量还揭示出大部分"空闲但突发"的工具沙箱,对这类沙箱而言,基于快照的挂起在经济上仍不可行。我们的安全分析进一步推动了攻击面最小化沙箱设计的必要性。基于这些洞察,我们讨论了对智能体原生服务系统的愿景,其特征包括:轨迹感知的可观测性、面向轨迹上下文管理与智能体感知沙箱弹性的控制面,以及使工具执行攻击面最小化的沙箱设计。

2 Background and Motivation / 背景与动机

EN Figure 1: Overview of conventional LLM-serving and emerging agent-serving systems. CN 图 1: 传统 LLM 服务与新兴智能体服务系统的概览。

We first discuss the differences between the conventional LLM serving and emerging agentic workloads, and their implications for modern cloud AI infrastructure. We then discuss the existing benchmarking frameworks and why they fall short for systems research in agentic systems.

我们首先讨论传统 LLM 服务与新兴智能体化工作负载之间的差异,及其对现代云 AI 基础设施的启示。随后讨论现有基准框架的不足,以及它们为何难以胜任智能体系统的系统研究。

2.1 From Model Requests to Agent Trajectories / 从模型请求到智能体轨迹

Conventional LLM serving systems are organized around the execution of individual model requests. As shown in Fig. 1a](https://arxiv.org/html/2607.29069v1#S2.F1)a), the serving runtime processes a prompt, generates output tokens, manages accelerator memory, and returns a response (Zhong et al., 2024; Yu et al., 2022). Scheduling and memory management are therefore scoped primarily to the active invocation. Depending on the cache policy, the engine may retain, offload, reuse, or reclaim the invocation’s KV-cache state after execution (Kwon et al., 2023; Pan et al., 2025; Zheng et al., 2024; Qin et al., 2025). This request-centric design naturally prioritizes metrics such as time to first token, time per output token, request latency, and aggregate token throughput (Kwon et al., 2023; Zhong et al., 2024; Patel et al., 2024; Hu et al., 2024b; Yu et al., 2022; Zheng et al., 2024).

传统 LLM 服务系统围绕单个模型请求的执行进行组织。如图 1a](https://arxiv.org/html/2607.29069v1#S2.F1)a) 所示,服务运行时处理提示词、生成输出 Token、管理加速器内存并返回响应(Zhong et al., 2024; Yu et al., 2022)。因此,调度与内存管理主要限定于当前激活的调用范围。根据缓存策略,引擎可以在执行后保留、卸载、复用或回收该调用的 KV 缓存状态(Kwon et al., 2023; Pan et al., 2025; Zheng et al., 2024; Qin et al., 2025)。这种以请求为中心的设计自然优先关注首 Token 延迟、每输出 Token 时间、请求延迟与聚合 Token 吞吐等指标(Kwon et al., 2023; Zhong et al., 2024; Patel et al., 2024; Hu et al., 2024b; Yu et al., 2022; Zheng et al., 2024)。

Agent serving departs from this invocation-centric abstraction. As shown in Fig. 1b](https://arxiv.org/html/2607.29069v1#S2.F1)b), the harness repeatedly invokes the LLM, dispatches tool calls to a sandboxed runtime, incorporates tool results into later prompts, and decides when the task is completed (Luo et al., 2026; Liu et al., 2024; Yao et al., 2022). A single user request may therefore span many dependent model and tool steps.

智能体服务则偏离了这种以调用为中心的抽象。如图 1b](https://arxiv.org/html/2607.29069v1#S2.F1)b) 所示,harness 反复调用 LLM、将工具调用分派到沙箱化运行时、将工具结果纳入后续提示词,并决定任务何时完成(Luo et al., 2026; Liu et al., 2024; Yao et al., 2022)。因此,单个用户请求可能跨越许多相互依赖的模型步骤与工具步骤。

In this paper, we define an agent trajectory as the ordered sequence of LLM invocations, tool calls, tool results, and harness decisions produced while serving one user request. Individual model requests are steps within this larger execution unit rather than independent transactions. Trajectory state persists across these steps. The harness may represent this state as message history or structured events, while the serving layer may materialize reusable portions as cached model state, such as KV-cache prefixes. At the same time, model-generated actions execute on the host side within an isolated sandbox. Agent serving consequently spans GPU inference, host-side orchestration, persistent context state, and sandboxed execution, with dependencies across all four components.

本文中,我们将 智能体轨迹(agent trajectory) 定义为服务单个用户请求过程中产生的 LLM 调用、工具调用、工具结果与 harness 决策的有序序列。单个模型请求只是这一更大执行单元中的步骤,而非独立事务。轨迹状态在这些步骤之间持续存在。harness 可能将该状态表示为消息历史或结构化事件,而服务层可能将可复用的部分物化为缓存的模型状态(如 KV 缓存前缀)。与此同时,模型生成的动作在主机侧于隔离沙箱内执行。因此,智能体服务横跨 GPU 推理、主机侧编排、持久化上下文状态与沙箱化执行四大组件,且这四个组件之间存在相互依赖。

2.2 Limitations of Existing Benchmarking Frameworks / 现有基准框架的局限

Existing benchmarks answer complementary questions at incompatible granularities. Capability-oriented benchmarks evaluate whether an agent completes a realistic task, but largely treat the execution path as a black box (Zhou et al., 2024; Gur et al., 2024; Jimenez et al., 2024; Merrill et al., 2026; Mialon et al., 2024; Xie et al., 2024; Yao et al., 2024). Systems benchmarks can measure end-to-end performance and resource behavior, but typically consider individual requests or transactions as their unit of analysis (Reddi et al., 2020; Ferdman et al., 2012; Gan et al., 2019; Ustiugov et al., 2021).

现有基准以互不兼容的粒度回答互补的问题。面向能力的基准评估智能体能否完成现实任务,但大多将执行路径视为黑盒(Zhou et al., 2024; Gur et al., 2024; Jimenez et al., 2024; Merrill et al., 2026; Mialon et al., 2024; Xie et al., 2024; Yao et al., 2024)。系统基准能够测量端到端性能与资源行为,但通常以单个请求或事务作为分析单元(Reddi et al., 2020; Ferdman et al., 2012; Gan et al., 2019; Ustiugov et al., 2021)。

None of the prior frameworks and benchmarks reveal how system behavior affects progress within a multi-step agent task. Task semantics are often entangled with harness- and runtime-specific choices, execution events remain fragmented across model, harness, and sandbox components, and stateful tool behavior depends on the underlying substrate. Agent serving therefore creates a measurement gap between semantic task progress and cross-layer system execution, making it difficult to isolate configuration effects, reconstruct complete trajectories, and attribute resource bottlenecks to task-level delays and failures.

以往的框架与基准都无法揭示系统行为如何影响多步骤智能体任务中的进度。任务语义往往与 harness 及运行时特定的选择纠缠在一起,执行事件在模型、harness 与沙箱各组件间呈碎片化分布,而有状态工具行为又依赖于底层基质。因此,智能体服务在"语义任务进度"与"跨层系统执行"之间形成了测量鸿沟,使得难以隔离配置影响、重建完整轨迹,以及将资源瓶颈归因于任务级延迟与失败。

3 Aries: Trajectory-Level Telemetry / 3 Aries:轨迹级遥测

EN Figure 2: Aries architecture. The blue-colored components are instrumentation in the agent-serving loop, dedicated to bridging the measurement gap.

CN 图 2: Aries 架构。蓝色组件是智能体服务循环中的插桩(Instrumentation),专门用于弥合测量鸿沟。

The goal of Aries is to bridge the measurement gap between semantic task outcomes and cross-layer system behavior by treating an agent trajectory as the first-class citizen. This entails three challenges. First, task semantics are often entangled with execution choices, hindering controlled comparison across stacks. Second, agent execution spans independently managed components, making trajectories difficult to reconstruct or attribute from local logs. Third, tool execution is both stateful and substrate-dependent, complicating consistent comparison of its behavior and resource cost. Aries addresses these challenges with separate task and execution specifications, a shared trajectory schema that correlates cross-layer telemetry, and a substrate-independent interface for stateful tool execution and instrumentation.

Aries 的目标是通过将智能体轨迹视为一等公民,弥合"语义任务结果"与"跨层系统行为"之间的测量鸿沟。这带来三项挑战。其一,任务语义常与执行选择纠缠,阻碍跨技术栈的受控对比。其二,智能体执行横跨独立管理的组件,使得仅凭本地日志难以重建或归因轨迹。其三,工具执行既有状态又依赖底层基质,使其行为与资源成本的对比难以一致。Aries 通过"任务与执行规格分离"“关联跨层遥测的共享轨迹模式"以及"面向有状态工具执行与插桩的、与底层基质无关的接口"来解决这些挑战。

Fig. 2 presents the architecture of Aries. Task and execution loaders validate the specifications; the runner coordinates the harness and accesses LLM backends and sandbox substrates through adapters; and the telemetry collector merges framework events with cross-layer telemetry into a unified record. This design separates task semantics from execution configuration, preserves cross-component provenance, and standardizes stateful tool execution across sandbox substrates.

图 2 展示了 Aries 的架构。任务加载器与执行加载器负责校验规格;运行器协调 harness,并通过适配器访问 LLM 后端与沙箱基底;遥测收集器将框架事件与跨层遥测合并为统一记录。这一设计将任务语义与执行配置分离,保留跨组件来源信息,并跨沙箱基底标准化有状态工具执行。

Preserving task semantics across configurations. Agent frameworks often embed task semantics within harness-specific logic, so replacing a runtime component may inadvertently change the task being evaluated. Aries therefore separates the task specification, which captures the workload and its success semantics, from the execution specification, which selects the harness, model backend, sandbox, and telemetry stack. Dedicated loaders validate the two specifications, while LLM backend and sandbox adapters bind them to a concrete execution stack without task-specific changes. This separation enables controlled comparisons across configurations.

跨配置保留任务语义。 智能体框架往往将任务语义嵌入 harness 特有的逻辑中,因此替换运行时组件可能无意中改变被评估的任务。为此,Aries 将任务规格(捕获工作负载及其成功语义)与执行规格(选择 harness、模型后端、沙箱与遥测技术栈)分离。专用加载器对两种规格进行校验,而 LLM 后端与沙箱适配器将其绑定到具体执行栈,无需针对任务的修改。这一分离支持跨配置的受控对比。

Reconstructing cross-component trajectories. Agent runs may contain overlapping, failed, or retried operations across independently managed components, making component-local logs and timestamps insufficient for reliable reconstruction and attribution. Aries propagates trajectory and event identifiers across model and tool boundaries and records causal and ordering metadata under a common event schema. The telemetry collector then associates component metrics with these events, revealing where delays and resource pressure arise along the trajectory.

重建跨组件轨迹。 智能体运行可能包含跨独立管理组件的重叠、失败或重试操作,使得组件本地日志与时间戳不足以进行可靠的重建与归因。Aries 在模型与工具边界传播轨迹与事件标识符,并在统一事件模式下记录因果与排序元数据。随后遥测收集器将组件指标关联到这些事件,从而揭示延迟与资源压力在轨迹中的产生位置。

Making stateful tool execution observable and portable. Tool calls operate on a persistent environment whose state affects subsequent trajectory steps, while different sandbox substrates expose different lifecycle and telemetry interfaces. Aries standardizes environment setup, tool invocation, state continuity, result collection, and telemetry hooks, while sandbox adapters map these operations onto concrete substrates. This makes tool execution a first-class, consistently observable stage of the trajectory while supporting different sandbox technologies.

使有状态工具执行可观测且可移植。 工具调用作用于一个持久化环境,其状态会影响后续轨迹步骤,而不同的沙箱基底暴露不同的生命周期与遥测接口。Aries 对环境设置、工具调用、状态连续性、结果收集与遥测挂钩进行标准化,同时沙箱适配器将这些操作映射到具体基底。这使得工具执行成为轨迹中一等公民的、一致可观测的阶段,同时支持不同的沙箱技术。

4 Why Is Modern Cloud AI Infrastructure a Poor Fit for Agents? / 为何现代云 AI 基础设施不适合智能体?

Using Aries, we evaluate whether request-centric modern cloud AI infrastructure is sufficient for agent serving. Specifically, we ask which components determine the end-to-end critical path, how long-lived trajectory context affects serving efficiency and accuracy, how resources are utilized across LLM backends and tool sandboxes, and the adequacy of existing tool-isolation mechanisms. We first analyze an eight-hour workday production trace, including request lengths from ten LLM-serving instances and one-minute CPU and memory samples from 100 agent sandboxes, to establish high-level workload behavior at deployment scale. As the operational traces do not expose synchronized cross-layer events, we then reproduce the observed patterns with open harnesses and datasets and use Aries for deeper system-level analysis.

利用 Aries,我们评估以请求为中心的现代云 AI 基础设施是否足以支撑智能体服务。具体而言,我们研究:哪些组件决定端到端关键路径;长生命周期轨迹上下文如何影响服务效率与精度;LLM 后端与工具沙箱之间的资源利用情况;以及现有工具隔离机制的充分性。我们首先分析一个八小时工作日的生产轨迹,包括来自十个 LLM 服务实例的请求长度,以及来自 100 个智能体沙箱的每分钟 CPU 与内存采样,以在部署规模上建立高层工作负载行为。由于运维轨迹未暴露同步的跨层事件,我们随后使用开源 harness 与数据集复现观测到的模式,并借助 Aries 开展更深入的系统级分析。

For this controlled analysis, Aries integrates OpenHands (Wang et al., 2026), Hermes Agent (Nous Research, 2026), and OpenClaw (OpenClaw Contributors, 2026), and evaluates serving tasks from SWE-Bench Pro (Deng et al., 2025), Terminal-Bench 2 (Team, 2025), and DeepResearch Bench (Du et al., 2025). We sample 20 tasks from each benchmark to achieve 50% task success rate (important for the accuracy studies) with OpenClaw and repeat each task five times. Unless otherwise stated, experiments use Qwen3.6-35B-A3B-FP8 (Qwen Team, 2026), served locally by SGLang (Zheng et al., 2024) on a 96-core node with one NVIDIA H100 GPU with 94 GB of HBM.

为进行这一受控分析,Aries 集成了 OpenHands(Wang et al., 2026)、Hermes Agent(Nous Research, 2026)与 OpenClaw(OpenClaw Contributors, 2026),并评估来自 SWE-Bench Pro(Deng et al., 2025)、Terminal-Bench 2(Team, 2025)与 DeepResearch Bench(Du et al., 2025)的服务任务。我们从每个基准各采样 20 个任务,使 OpenClaw 达到 50% 的任务成功率(这对精度研究很重要),并每个任务重复五次。除非另有说明,实验使用由 SGLang(Zheng et al., 2024)在本地提供的 Qwen3.6-35B-A3B-FP8(Qwen Team, 2026),运行于一个 96 核节点上,配备一块 94 GB HBM 的 NVIDIA H100 GPU。

4.1 Harnesses & Tools Matter as Much as Models / Harness 与工具与模型同等重要

Existing LLM-serving engines optimize token-centric metrics such as TTFT and TPOT (Zhong et al., 2024; Kwon et al., 2023; Zheng et al., 2024). These metrics omit the host-side intervals during which the harness processes model outputs or waits for tool execution, even though the agent cannot advance until these steps complete.

现有 LLM 服务引擎优化的是以 Token 为中心的指标,如 TTFT 与 TPOT(Zhong et al., 2024; Kwon et al., 2023; Zheng et al., 2024)。这些指标忽略了 harness 处理模型输出或等待工具执行的主机侧时间间隔,尽管在完成这些步骤之前智能体无法继续推进。

EN Figure 3: End-to-end average latency breakdown. Harness and tool execution contribute latency on par with LLM calls. CN 图 3: 端到端平均延迟分解。harness 与工具执行带来的延迟与 LLM 调用相当。

EN Figure 4: Latency distributions for LLM inference vs. tool execution. Tool calls exhibit a severe long tail spanning multiple orders of magnitude. CN 图 4: LLM 推理与工具执行的延迟分布。工具调用呈现出跨越多个数量级的严重长尾。

As the per-step timing information is too sensitive to collect in the commercial cluster, we base the analysis of the utility of existing metrics on the setup with public datasets and harnesses which record these events under a shared trajectory identifier and time base.

由于逐步计时信息在商业集群中过于敏感而难以收集,我们基于公开数据集与 harness(它们在共享的轨迹标识符与时间基准下记录这些事件)来分析现有指标的效用。

Our profiling highlights two key system insights: (1) Critical-Path Shift to Harness and Tool Execution: As shown in Fig. 3, tool execution is an important latency contributor, contributing from 13% to 48% of the total latency across datasets and harnesses, while reaching 48% for Hermes Agent on Terminal-Bench 2. Beyond tool execution, the harness spends up to 9% of end-to-end time on orchestration and other host-side processing, further extending the interval between consecutive LLM calls and potentially prolonging the residency of trajectory state in GPU memory. (2) Long-Tailed Latency Profiles: Fig. 4 shows that while LLM latency remains important, tool calls exhibit a severe long tail that spans multiple orders of magnitude. Together, these results show that these massive host-side delays frequently dominate execution time, meaning that solely optimizing accelerator token generation cannot resolve the actual agent bottlenecks.

我们的剖析揭示了两个关键的系统洞察:(1) 关键路径向 harness 与工具执行转移: 如图 3 所示,工具执行是重要的延迟贡献者,在不同数据集与 harness 下贡献了总延迟的 13% 至 48%,其中 Hermes Agent 在 Terminal-Bench 2 上达到 48%。除工具执行外,harness 还花费高达 9% 的端到端时间用于编排及其他主机侧处理,进一步拉长了连续 LLM 调用之间的间隔,并可能延长轨迹状态在 GPU 内存中的驻留时间。(2) 长尾延迟分布: 图 4 表明,虽然 LLM 延迟依然重要,但工具调用呈现出跨越多个数量级的严重长尾。综合来看,这些结果说明,大量主机侧延迟常常主导执行时间,这意味着仅优化加速器的 Token 生成无法解决真实的智能体瓶颈。

EN Takeaway 1: End-to-end task duration depends heavily on harness and tool sandbox execution, rather than model inference alone. CN 结论 1: 端到端任务时长在很大程度上取决于 harness 与工具沙箱的执行,而不仅仅是模型推理。

4.2 Long Context Dictates Efficiency & Accuracy / 4.2 长上下文决定效率与精度

In conventional LLM serving, context lifetimes are typically scoped to the request-response cycle, allowing engines to reclaim KV cache pages post-query unless explicitly retained or offloaded for reuse (Kwon et al., 2023; Zheng et al., 2024; Zhong et al., 2024; Hu et al., 2024a; Qin et al., 2025; Liu et al., 2025). Conversely, long-horizon agents generate monotonically growing histories that persist across the entire trajectory. While existing frameworks apply semantic compression to fit system limits (Rasmussen et al., 2025; Xu et al., 2026; Chhikara et al., 2025), unmanaged context expansion still locks up physical GPU token pools and reduces serving concurrency (Li et al., 2025; Kariyappa and Suh, 2026). Retaining excessive history can increase tail delay, while aggressive compression may discard information required by later steps and reduce task accuracy (Kang et al., 2025; Kariyappa and Suh, 2026). Resolving this accuracy-efficiency tension requires shifting from passive memory allocation to trajectory-aware state management that balances the resource cost and future utility of retained context.

在传统 LLM 服务中,上下文生命周期通常限定于请求—响应循环,使引擎能在查询结束后回收 KV 缓存页,除非显式保留或卸载以复用(Kwon et al., 2023; Zheng et al., 2024; Zhong et al., 2024; Hu et al., 2024a; Qin et al., 2025; Liu et al., 2025)。相反,长时程智能体生成单调增长的历史记录,并贯穿整个轨迹持续存在。尽管现有框架采用语义压缩以适配系统限制(Rasmussen et al., 2025; Xu et al., 2026; Chhikara et al., 2025),但不受管理的上下文扩张仍会锁定物理 GPU Token 池,并降低服务并发度(Li et al., 2025; Kariyappa and Suh, 2026)。保留过多历史可能增加尾延迟,而激进压缩可能丢弃后续步骤所需的信息,从而降低任务精度(Kang et al., 2025; Kariyappa and Suh, 2026)。化解这一精度—效率张力,需要从被动的内存分配转向轨迹感知的状态管理,在保留上下文的资源成本与其未来效用之间取得平衡。

We first compare the context lengths observed in agentic serving and conventional inference, both retrieved from the production platform. Then, we characterize the trade-offs associated with the context budget and how they affect the accuracy and efficiency of agentic serving with the public harnesses and datasets. In the latter analysis, across all evaluated benchmarks, we sweep the context budget for both OpenClaw and Hermes Agent (omitting OpenHands for brevity) to determine when additional retained history stops providing meaningful accuracy gains. We then quantify the cost of context retention when serving 32 concurrent active agentic sessions with a 256K context size, sampling backend resource utilization once per second.

我们首先对比生产平台上智能体服务与传统推理中观测到的上下文长度。随后,我们刻画与上下文预算相关的权衡,以及它们如何借助公开 harness 与数据集影响智能体服务的精度与效率。在后一分析中,我们针对所有被评估的基准,对 OpenClaw 与 Hermes Agent 的上下文预算进行扫描(为简洁起见省略 OpenHands),以确定保留额外历史在何时不再带来有意义的精度提升。随后,我们在 256K 上下文大小下服务 32 个并发活跃智能体会话,以每秒一次的频率采样后端资源利用率,量化上下文保留的成本。

EN Figure 5: Context length and resident capacity comparison across conversation and agent requests in the production cluster. Agent workloads consume larger per-request KV footprints than conventional serving, sharply reducing the number of processed requests in parallel by an LLM backend. CN 图 5: 生产集群中对话请求与智能体请求的上下文长度与驻留容量对比。智能体工作负载的每请求 KV 占用比传统服务更大,大幅降低了 LLM 后端可并行处理的请求数。

EN Figure 6: Compaction and context overflow counts with different context window sizes, and the corresponding task success rate. Larger windows ease overflow/compaction pressure, but task success plateaus past workload-specific thresholds. CN 图 6: 不同上下文窗口大小下的压缩与上下文溢出次数,及对应的任务成功率。更大的窗口缓解了溢出/压缩压力,但任务成功率在工作负载特定阈值后趋于平缓。

We make three key observations: (1) Trajectory State Amplification: Production traces reveal substantially larger agent contexts than those observed in conventional inference workloads, which matches our results with the public datasets and harnesses. As quantified in Fig. 5, the average agent context consumes a substantially larger KV footprint than conventional inference. Under a fixed 25GB budget with the Qwen3.6 model, this state amplification reduces the maximum resident capacity from over 176 concurrent serving requests to merely 40—a 4.4× reduction. (2) Context Sufficiency Thresholds: Using the open datasets and harnesses, we show that long-context retention improves accuracy only up to a task-specific threshold, beyond which additional history, albeit consuming more memory, provides little to no accuracy benefits. As shown in Fig. 6, OpenClaw on DeepResearch Bench reaches peak success (95%) at 160K context, coinciding with the elimination of context overflows. Conversely, Terminal-Bench 2 and SWE-Bench Pro plateau or degrade once overflow pressure is removed, performing best at small or mid-size windows rather than the maximum context size. (3) Capacity-induced Batch Limitation: our results with public datasets further show that large retained state severely constrains serving capacity. Fig. and Fig. show that memory pressure limits effective batching despite a 32-session backlog; token-pool utilization peaks at 97%, while the engine sustains a median of only 22 concurrent requests. Under this pressure, the scheduler preempts active sessions, and the p95 internal queueing delay reaches 7.9s (Fig. ).

我们得到三点关键观察:(1) 轨迹状态放大: 生产轨迹显示,智能体上下文显著大于传统推理工作负载中观测到的上下文,这与我们在公开数据集与 harness 上的结果一致。如图 5 所量化,平均智能体上下文消耗的 KV 占用显著大于传统推理。在 Qwen3.6 模型、固定 25GB 预算下,这种状态放大使最大驻留容量从 176 个并发服务请求骤降至仅 40 个——即 4.4× 的缩减。(2) 上下文充分性阈值: 借助开源数据集与 harness,我们证明长上下文保留仅在达到任务特定阈值前提升精度,超过该阈值后,额外历史尽管消耗更多内存,却几乎不再带来精度收益。如图 6 所示,OpenClaw 在 DeepResearch Bench 上于 160K 上下文处达到峰值成功率(95%),此时恰好消除了上下文溢出。相反,Terminal-Bench 2 与 SWE-Bench Pro 在溢出压力消除后趋于平缓甚至下降,其最佳性能出现在较小或中等窗口,而非最大上下文。(3) 容量导致的批处理受限: 我们在公开数据集上的结果进一步表明,大规模保留状态严重制约服务容量。图中数据表明,尽管存在 32 个会话的积压,内存压力仍限制了有效批处理;Token 池利用率峰值达到 97%,而引擎仅维持中位数 22 个并发请求。在此压力下,调度器抢占活跃会话,p95 内部排队延迟达到 7.9 秒。

EN Figure 7: CDFs over the one-hour window of the agent serving. The workload sustains high concurrency and KV pressure, while long waits are concentrated in the request tail. CN 图 7: 智能体服务一小时窗口内的 CDF(累积分布函数)。工作负载维持高并发与高 KV 压力,而长时间等待集中在请求尾部。 EN Takeaway 2: Retaining context beyond its task-dependent utility yields diminishing accuracy benefits while increasing KV-cache occupancy, limiting batching, and prolonging queueing delays. The system must therefore actively manage trajectory state to balance task accuracy against serving efficiency. CN 结论 2: 保留超出任务依赖效用的上下文,只会带来递减的精度收益,同时增加 KV 缓存占用、限制批处理并延长排队延迟。因此,系统必须主动管理轨迹状态,以在任务精度与服务效率之间取得平衡。

4.3 Tool Sandboxes Must Be Elastic / 工具沙箱必须具有弹性

Agent serving introduces CPU-based processing to the critical path: it alternates between idle intervals during LLM inference and short CPU/memory bursts during tool execution. Static peak resource provisioning avoids throttling these tool calls but leaves resources stranded a significant portion of the time. Cloud systems such as AWS Lambda MicroVMs (Amazon Web Services, 2026a) and Google Agent Sandbox (Google Cloud, 2026) attempt to reduce waste by applying a stateful scale-to-zero approach via sandbox snapshot capture and restore (C/R). However, agentic tool execution patterns lead to a high number of capture-restore cycles, incurring additional costs, as we show below.

智能体服务将基于 CPU 的处理引入关键路径:它在 LLM 推理期间的空闲间隔与工具执行期间短暂的 CPU/内存突发之间交替。静态的峰值资源供给虽能避免限制这些工具调用,却会在相当长的时间内造成资源闲置。AWS Lambda MicroVMs(Amazon Web Services, 2026a)与 Google Agent Sandbox(Google Cloud, 2026)等云系统试图通过"沙箱快照捕获与恢复(C/R)“实现 有状态缩至零(scale-to-zero) 来减少浪费。然而,智能体化工具执行模式会导致大量捕获—恢复周期,带来额外成本,我们将在下文中展示。

EN Figure 8: CDFs of Harness CPU and memory utilization aggregating measurements taken every minute from the production cluster. The values are normalized for trace anonymization purposes. CN 图 8: 生产集群中每分钟聚合测量的 harness CPU 与内存利用率 CDF。数值已归一化以进行轨迹匿名化。

We collect CPU and memory utilization from over 100 sandboxes on the same commercial platform, each co-hosting an OpenClaw-style harness and its tool calls. To confirm the trend and for in-depth analysis, we also collect CPU and memory utilization of harness and tool sandboxes in a controlled environment every second and analyze their distributions for OpenClaw and Hermes using the cloud DeepSeek V4 Flash API, which is production-grade and highly stable for tool call generation accuracy. We then analytically replay the recorded tool-call traces against the AWS Lambda MicroVMs pricing model (Amazon Web Services, 2026b) under different sandbox keepalive policies as in serverless deployments (Shahrad et al., 2020; Fuerst and Sharma, 2021). We report the instance cost (CPU and memory usage), the snapshot C/R cost, and their sum. The ∞ keepalive point corresponds to the persistent sandbox. DeepResearch is excluded from this comparison due to the tools’ low CPU usage.

我们收集同一商业平台上超过 100 个沙箱(每个沙箱共托管一个 OpenClaw 风格 harness 及其工具调用)的 CPU 与内存利用率。为确认趋势并进行深入分析,我们还在受控环境中以每秒一次的频率收集 harness 与工具沙箱的 CPU 和内存利用率,并使用云端 DeepSeek V4 Flash API(生产级、对工具调用生成精度高度稳定)分析 OpenClaw 与 Hermes 的分布。随后,我们参照无服务器部署中的不同沙箱保活策略(Shahrad et al., 2020; Fuerst and Sharma, 2021),针对 AWS Lambda MicroVMs 计费模型(Amazon Web Services, 2026b)对记录的工具调用轨迹进行解析式重放。我们报告实例成本(CPU 与内存使用)、快照 C/R 成本及其总和。保活时间为无穷大的点对应持久沙箱。由于 DeepResearch 工具 CPU 占用较低,本对比将其排除。

EN Figure 9: Tool and harness CPU and memory utilization CDFs aggregating measurements taken every second. The tool environment is mostly idle, with high CPU utilization spikes during tool calls. CN 图 9: 每秒聚合测量的工具与 harness CPU 与内存利用率 CDF。工具环境大部分时间处于空闲,在工具调用期间出现高 CPU 利用率尖峰。

EN Figure 10: Absolute customer-visible cost of snapshot-based stateful scale-to-zero under different keepalive timeouts based on AWS Lambda MicroVM billing model. Short keepalive reduces instance cost but triggers frequent snapshot C/R, while longer keepalive reduces C/R by keeping the sandbox resident for more idle time. CN 图 10: 基于 AWS Lambda MicroVM 计费模型,不同保活超时下基于快照的有状态缩至零的绝对客户可见成本。较短的保活时间降低实例成本,但触发频繁的快照 C/R;较长的保活时间通过让沙箱在更多空闲时间内保持驻留来减少 C/R。

Fig. 8 shows that CPU compute and memory utilization in the production setting is spiky. We reproduce this with the public datasets and harnesses (Fig. 9), showing that agent tool sandboxes spend most of their trajectories at low utilization, yet exhibit sharp spikes in CPU and memory demand during tool execution. Across harnesses and workloads, CPU and memory remain idle for more than 80% and 50% of the time, respectively. At the same time, the tool-call-conditioned distributions show much more intensive resource use: the portion of time with idle resources drops below 20% for both CPU and memory. Fig. 10 shows that both users and cloud providers have an incentive to reduce keepalive to lower their costs: users avoid paying for idle sandboxes, while providers reclaim stranded capacity sooner. Yet current snapshot-based mechanisms and pricing prevent this alignment: if tool sandboxes are torn down after each tool call, their cost falls by 65%, but repeated snapshot C/R raises the total customer cost to 4.9× compared to the persistent baseline. Keeping tool sandboxes for longer reduces C/R overhead but increases idle time. Thus, today’s system forces a false choice between paying for idle capacity and paying for repeated snapshots and restores. However, these results suggest that the total cost can be reduced by 3× in an ideal system compared to the persistent-sandbox baseline.

图 8 显示,生产环境中的 CPU 计算与内存利用率呈尖峰状。我们使用公开数据集与 harness 复现了这一现象(图 9),表明智能体工具沙箱在轨迹的大部分时间里利用率较低,却在工具执行期间呈现出](https://arxiv.org/html/2607.29069v1#S4.F9)),%E8%A1%A8%E6%98%8E%E6%99%BA%E8%83%BD%E4%BD%93%E5%B7%A5%E5%85%B7%E6%B2%99%E7%AE%B1%E5%9C%A8%E8%BD%A8%E8%BF%B9%E7%9A%84%E5%A4%A7%E9%83%A8%E5%88%86%E6%97%B6%E9%97%B4%E9%87%8C%E5%88%A9%E7%94%A8%E7%8E%87%E8%BE%83%E4%BD%8E,%E5%8D%B4%E5%9C%A8%E5%B7%A5%E5%85%B7%E6%89%A7%E8%A1%8C%E6%9C%9F%E9%97%B4%E5%91%88%E7%8E%B0%E5%87%BA) CPU 与内存需求的急剧尖峰。在各类 harness 与工作负载中,CPU 与内存在超过 80% 与 50% 的时间里处于空闲。与此同时,以工具调用为条件的分布显示出更密集的资源使用:CPU 与内存的空闲时间占比均降至 20% 以下。图 10 表明,用户与云提供商都有动力缩短保活时间以降低成本:用户避免为空闲沙箱付费,而提供商能更早回收闲置容量。然而,当前的基于快照的机制与定价阻碍了这种一致性:如果在每次工具调用后拆除工具沙箱,其成本下降 65%,但反复的快照 C/R 使客户总成本相比持久基线升至 4.9 倍。将工具沙箱保留更久可降低 C/R 开销,但会增加空闲时间。因此,当今系统迫使人们在"为闲置容量付费"与"为反复快照与恢复付费"之间做出错误取舍。不过,这些结果表明,在理想系统中,总成本相比持久沙箱基线可降低 3 倍。

EN Takeaway 3: Agent sandboxes are mostly idle but bursty when tools execute. Both users and providers should benefit from idle resource reclamation, yet current state-management mechanisms and pricing make aggressive suspension uneconomical. CN 结论 3: 智能体沙箱大部分时间处于空闲,但在工具执行时呈突发性。用户与提供商都应从闲置资源回收中获益,然而当前的状态管理机制与定价使激进式挂起不具经济性。

4.4 Tool Sandboxes Are Under Continuous Siege / 工具沙箱正面临持续围攻

Cloud infrastructure is already designed to handle untrusted code execution, and uses the same mechanisms to sandbox agentic workloads. Beyond LLM-specific challenges, e.g., KV caches side-channels (Chu et al., 2025) or sensitive input data (Luo et al., 2025), agents have the ability to autonomously discover and exploit vulnerabilities (Ullah et al., 2025; Zhang et al., 2026; Xu et al., 2025). Frontier models already found advanced vulnerabilities in production system software, such as OS kernels and VMMs (Carlini et al., 2026), hinting at the possibility of AI agents breaching cloud isolation.

云基础设施本已为处理不可信代码执行而设计,并使用同样的机制对智能体化工作负载进行沙箱化。除 LLM 特有的挑战(如 KV 缓存侧信道(Chu et al., 2025)或敏感输入数据(Luo et al., 2025))外,智能体还具备自主发现并利用漏洞的能力(Ullah et al., 2025; Zhang et al., 2026; Xu et al., 2025)。前沿模型已能在操作系统内核、VMM 等生产系统软件中发现高级漏洞(Carlini et al., 2026),这预示着 AI 智能体存在突破云隔离的可能性。

We use the share of Linux kernel CVEs identified by AI tools as a proxy for the ability of AI agents to find vulnerabilities autonomously. We compute that share as the proportion of commits in the Linux CVEs that explicitly mention using AI tools to find and fix vulnerabilities. We display our findings in Fig. 11.

我们以"AI 工具发现的 Linux 内核 CVE 占比"作为衡量 AI 智能体自主发现漏洞能力的代理指标。我们将该占比定义为 Linux CVE 中明确提及使用 AI 工具发现并修复漏洞的提交所占的比例。我们的发现展示于图 11。](https://arxiv.org/html/2607.29069v1#S4.F11)%E3%80%82)

EN Figure 11: Total and relative share of Linux kernel CVEs reported as found by AI agents or with AI assistance. There is a sharp increase in AI-found CVEs in Q2 of 2026. CN 图 11: 报告由 AI 智能体或借助 AI 辅助发现的 Linux 内核 CVE 的总数与相对占比。2026 年第二季度,AI 发现的 CVE 急剧增加。

AI tools are playing an increasing role in finding vulnerabilities. On the Linux kernel, this is especially true since the introduction of Sashiko, an agentic patch review system (Gushchin, 2026). The trend holds across CVE publishers, with a 3.5× increase in published CVEs for the second quarter of 2026 compared to previous records (Emberson, 2026). Exploiting known CVEs is also becoming cheap: agentic systems can develop exploits for a few dollars (Ullah et al., 2025). Agentic workloads are already highly capable of carrying sophisticated attacks against the cloud infrastructure. Therefore, sandboxes should expose a minimal attack surface, but also rely on defense-in-depth mechanisms to anticipate latent vulnerabilities.

AI 工具在发现漏洞中扮演着越来越重要的角色。在 Linux 内核上,自引入智能体化补丁审查系统 Sashiko 以来尤其如此(Gushchin, 2026)。这一趋势在各大 CVE 发布方中均成立,2026 年第二季度已发布的 CVE 相比此前的纪录增长了 3.5 倍(Emberson, 2026)。利用已知 CVE 也日益廉价:智能体系统仅需几美元即可开发出利用代码(Ullah et al., 2025)。智能体化工作负载已具备对云基础设施发起复杂攻击的高度能力。因此,沙箱应暴露最小的攻击面,同时还应依赖纵深防御机制以应对潜在漏洞。

EN Takeaway 4: AI agents are highly capable of autonomously finding and exploiting vulnerabilities. Tool sandboxes should be designed to withstand advanced attacks and to anticipate increasing offensive model capabilities by reducing the attack surface. CN 结论 4: AI 智能体具备高度的自主发现与利用漏洞的能力。工具沙箱的设计应能抵御高级攻击,并通过缩小攻击面来预期不断增强的进攻性模型能力。

5 Future Research Directions / 未来研究方向

EN Figure 12: Agent-native AI serving system vision. CN 图 12: 智能体原生 AI 服务系统愿景。

Our results suggest that agent-serving systems should coordinate optimization decisions across the trajectory rather than manage model inference, context state, and tool execution independently. As illustrated in Fig. 12, this requires jointly optimizing task progress, capacity, and resources through four co-designed directions across the Runtime, Control, and Execution layers: (1) trajectory-capturing metrics spanning all layers; (2) control plane for trajectory-aware context management; (3) elastic tool execution management via sandbox control; and (4) minimizing the tool sandbox attack surface.

我们的结果表明,智能体服务系统应跨轨迹协调优化决策,而不是独立地管理模型推理、上下文状态与工具执行。如图 12 所示,这需要通过四个协同设计方向,在运行时(Runtime)、控制面(Control)与执行(Execution)三个层面联合优化任务进度、容量与资源:(1) 覆盖所有层的轨迹捕获指标;(2) 面向轨迹感知上下文管理的控制面;(3) 通过沙箱控制实现弹性工具执行管理;(4) 最小化工具沙箱攻击面。

5.1 Trajectory-capturing Metrics / 轨迹捕获指标

Future agent-serving metrics should continuously quantify how system execution contributes to task progress. As illustrated in Fig. 13, evaluation should connect end-to-end outcomes, such as task success and completion rate, with low-level measures of execution speed and cost, including latency, generated tokens, and resource consumption. Task goodput and solved tasks per generated token provide useful endpoint metrics, but they become available only after a trajectory completes and cannot guide runtime decisions.

未来的智能体服务指标应持续量化系统执行对任务进度的贡献。如图 13 所示,评估应将任务成功率、完成率等端到端结果,与延迟、生成 Token 数、资源消耗等执行速度与成本的底层度量相连接。任务有效吞吐与"每生成 Token 解决的已解决问题数"提供了有用的端点指标,但它们仅在轨迹完成后才可用,无法指导运行时决策。

A key research direction is therefore to derive online utility signals from trajectory events. Such signals should estimate the incremental progress contributed by each model call, harness decision, and tool interaction while accounting for its time and resource cost. They should distinguish productive actions from repeated or non-advancing steps and aggregate into end-to-end measures of task accuracy, execution speed, and system cost. This connection would allow low-level system optimizations to be evaluated by their actual contribution to agent task completion.

因此,一个关键研究方向是从轨迹事件中推导在线效用信号。这类信号应估计每次模型调用、harness 决策与工具交互所带来的增量进度,同时计入其时间与资源成本。它们应区分有效动作与重复或非推进性步骤,并聚合为任务精度、执行速度与系统成本的端到端度量。这种连接将使得底层系统优化能够依据其对智能体任务完成的实际贡献进行评估。

EN Figure 13: Token throughput, solved tasks per 1 million generated tokens, and solved tasks per hour. Token throughput can diverge from task goodput and execution efficiency in agent serving. CN 图 13: Token 吞吐、每 100 万生成 Token 解决的已解决问题数,以及每小时解决的已解决问题数。在智能体服务中,Token 吞吐可能与任务有效吞吐及执行效率相背离。

5.2 Control Plane for Trajectory-Aware Context Management / 面向轨迹感知上下文管理的控制面

Our measurements in §4.2 reveal a workload-dependent trade-off between context sufficiency and serving capacity. Additional context increases KV-cache pressure and queueing delay, even when it provides only a small improvement in task success. This motivates a trajectory-aware control plane that monitors task and memory states, determines how context should be retained and placed, and directs the LLM backend to perform context compaction, KV-cache compression, or state transfer. (1) Adaptive Context Retention. The control plane should estimate the future utility of context as a trajectory evolves and request compaction or compression when the expected accuracy benefit of retained state no longer justifies its memory cost. Such decisions must remain conservative enough to preserve information needed by later model invocations. (2) Hierarchical State Placement. The control plane should coordinate long-lived context placement across accelerator and lower-cost memory tiers rather than leaving all state resident on the accelerator. It should trigger KV-cache migration when the released accelerator capacity outweighs the transfer and restoration overhead.

我们在第 4.2 节的测量揭示了"上下文充分性"与"服务容量"之间随工作负载变化的权衡。额外上下文会增加 KV 缓存压力与排队延迟,即便其对任务成功率的提升微乎其微。这推动了轨迹感知控制面的诞生:它监控任务与内存状态,决定上下文应如何保留与放置,并指导 LLM 后端执行上下文压缩、KV 缓存压缩或状态迁移。(1) 自适应上下文保留。 控制面应随轨迹演进估计上下文的未来效用,并在保留状态的预期精度收益不再抵偿其内存成本时请求压缩或压缩处理。此类决策必须足够保守,以保留后续模型调用所需的信息。(2) 分层状态放置。 控制面应协调长生命周期上下文在加速器与低成本内存层级之间的放置,而非让所有状态驻留在加速器上。当释放的加速器容量超过迁移与恢复开销时,它应触发 KV 缓存迁移。

5.3 Control Plane for Agent-Aware Elasticity / 面向智能体感知弹性的控制面

Our characterization in §4.3 shows that host-side resource demand is highly intermittent. Static provisioning wastes capacity, while conventional, horizontal autoscaling mechanisms are too expensive at such granularity. Future agent-serving platforms therefore require an elasticity control plane designed around model-driven, stateful execution. (1) Vertical, Model-Aware Compute Management. Research should explore how signals from the model, harness, and runtime can guide timely vertical scaling of individual sandboxes to satisfy the identified volatile demand for memory and core count fluctuations. Key questions include which signals reliably predict demand, how uncertainty should be handled, and how resources should be allocated across competing trajectories. (2) State Management for Elastic Execution. Because sandboxes retain files, processes, caches, and network state, elasticity requires decoupling execution state from physical placement and resource allocation. Research is needed on what state must remain resident, what can be externalized or reconstructed, and how to preserve consistency while resizing, suspending, or relocating sandboxes.

我们在第 4.3 节的刻画表明,主机侧资源需求高度间歇。静态供给浪费容量,而传统水平自动扩缩机制在此粒度下过于昂贵。因此,未来的智能体服务平台需要围绕"模型驱动的有状态执行"设计的弹性控制面。(1) 垂直的、模型感知的计算管理。 研究应探索如何利用来自模型、harness 与运行时的信号,及时指导单个沙箱的垂直扩缩,以满足已识别的内存与核数波动的波动性需求。关键问题包括:哪些信号能可靠预测需求、如何处理不确定性,以及如何在相互竞争的轨迹之间分配资源。(2) 面向弹性执行的状态管理。 由于沙箱保留文件、进程、缓存与网络状态,弹性要求将执行状态与物理放置及资源分配解耦。需要研究:哪些状态必须保持驻留、哪些可以外部化或重建,以及在调整大小、挂起或迁移沙箱时如何保持一致性。

5.4 Sandboxes with a Minimal Attack Surface / 攻击面最小的沙箱

AI agents are increasingly capable of autonomously finding and exploiting vulnerabilities. Thus, cloud infrastructure should anticipate increasingly sophisticated adversarial workloads. (1) Defense-in-Depth and Minimizing Attack Surface. Sandboxes should be tailored to the workload to reduce the attack surface by restricting tool, network, and filesystem access based on the target tasks. Tool execution should leverage both host-side VM-level isolation and guest-side kernel-level sandboxing. For instance, workloads such as deep research do not need direct bash access, but can be granted a sandboxed JavaScript runtime within the guest. Also, the system design has to minimize the attack surface exposed to side-channel timing attacks across the agentic stack. (2) Formal Security Guarantees. In the long term, formal verification of security properties is the only path to certifying the absence of vulnerabilities. Migration to verified systems, such as seL4 (Klein et al., 2009), as a host hypervisor or guest kernel, will require major but necessary system overhaul to ensure the cloud remains secure despite models’ increasing offensive capabilities.

AI 智能体正日益具备自主发现与利用漏洞的能力。因此,云基础设施应预期日益复杂的对抗性工作负载。(1) 纵深防御与最小化攻击面。 沙箱应针对工作负载量身定制,通过根据目标任务限制工具、网络与文件系统访问来缩小攻击面。工具执行应同时利用主机侧 VM 级隔离与客户机侧内核级沙箱化。例如,深度研究等工作负载无需直接 bash 访问,可授予其客户机内沙箱化的 JavaScript 运行时。此外,系统设计还必须最小化智能体化技术栈中暴露给侧信道时序攻击的攻击面。(2) 形式化安全保障。 长远来看,对安全属性的形式化验证是认证"不存在漏洞"的唯一途径。向 seL4(Klein et al., 2009)等已验证系统迁移(作为主机虚拟机监控程序或客户机内核),将需要重大但必要的系统重构,以确保在模型进攻能力不断增强的情况下云依然安全。

6 Conclusion / 结论

Agent serving turns conventional LLM serving into long-lived, stateful trajectories spanning model execution, harness logic, and sandboxes. We present Aries, a telemetry framework that combines reproducible experiments with production traces to characterize task and system behavior. Our findings motivate trajectory-capturing metrics, control planes for context management and agent-aware elasticity, and sandboxes with minimal attack surfaces.

智能体服务将传统 LLM 服务转变为横跨模型执行、harness 逻辑与沙箱的长期有状态轨迹。我们提出 Aries——一个将可复现实验与生产轨迹相结合以刻画任务与系统行为的遥测框架。我们的发现推动了轨迹捕获指标、面向上下文管理与智能体感知弹性的控制面,以及攻击面最小化沙箱的发展。

2.5 - 论文分析

论文内容分析

Aries 论文解读

这篇论文的全称是 Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework。它不是一篇给出新调度算法或新推理引擎的系统论文，而是一篇 测量 + 问题重构 的论文：先造一个能看清智能体全栈行为的实验框架 Aries，再用生产轨迹和开源 harness 证明——今天这套以请求为中心的云 AI 基础设施，从指标、上下文、沙箱弹性到隔离模型，都不适合智能体服务。

一句话概括其主张：智能体服务的基本执行单元已经从“一次模型调用”变成了“一条跨模型、harness、持久上下文和工具沙箱的轨迹”；继续用 TTFT / TPOT / Token 吞吐去优化，会把真正决定任务进度的瓶颈优化掉。

图 1 是全文的坐标系。左边是传统 LLM 服务：提示词进、Token 出，调度和显存管理都围着当前这一次调用转。右边是智能体服务：harness 反复调模型、把工具打进沙箱、把结果写回后续提示词，并决定任务何时结束。一次用户请求可以跨越许多相互依赖的模型步与工具步。后文所有测量，都是在问：如果观测单元从请求换成轨迹，现有云基础设施会在哪些地方失效。

1. 它站在哪一层，回答哪一类问题

先把 Aries 放到栈上，避免把它理解成生产调度器或沙箱产品。

任务规格（做什么、何谓成功）          ← 与执行解耦
执行规格（harness / 模型 / 沙箱 / 遥测）
        ↓
Aries 运行器
  → LLM 后端适配器（如 SGLang）
  → harness（OpenHands / Hermes / OpenClaw）
  → 沙箱适配器（异构基底上的有状态工具执行）
        ↓
遥测收集器：把跨组件事件合成一条轨迹

Aries 的位置是 实验与测量层。它不替代 vLLM / SGLang，也不替代 AWS Lambda MicroVM 或 Google Agent Sandbox。它要补的是这些组件各自为政之后消失的那张图：任务有没有在推进、延迟和资源压力到底出在轨迹的哪一段。

论文认为现有评测正好切在这条缝的两边，谁也连不上：

评测类型 回答的问题 看不见的东西
能力基准（SWE-Bench、Terminal-Bench、DeepResearch 等） 任务最终对不对 执行路径是黑盒，无法把失败归因到 harness、排队还是沙箱
系统基准（请求延迟、函数冷启动、事务吞吐） 单次调用或无状态函数快不快 观测单元不是多步轨迹，连不上任务进度

于是出现测量鸿沟：语义上的任务进度，和跨层的系统执行，没有共同的观测对象。Aries 把这个对象定义为 智能体轨迹（agent trajectory）——服务单个用户请求过程中，LLM 调用、工具调用、工具结果与 harness 决策的有序序列。单个模型请求只是这条更大执行单元里的一步，不是独立事务。

这一点看起来像定义游戏，实际上决定了后面所有结论能不能成立。如果还按“一次 generate”计费、调度、评测，第 4 节测到的 harness 延迟、KV 占用膨胀、沙箱空闲突发，都会被当成应用层噪音而不是服务系统的一等问题。

2. Aries 框架：三刀切开测量鸿沟

图 2 里蓝色组件是专门为弥合测量鸿沟插进去的。框架本身有三项设计，对应三项挑战。

第一刀：任务规格与执行规格分离。
智能体框架常把“这道题怎么算成功”写进某个 harness 的逻辑里。换一个运行时组件，评测的任务可能在不知不觉中被换掉。Aries 把工作负载及其成功语义做成任务规格，把 harness、模型后端、沙箱、遥测栈做成执行规格，两边独立校验、再由适配器绑到具体栈。没有这一刀，就无法做“同一任务、不同沙箱/不同模型”的受控对比——而智能体系统研究最缺的就是这种对比。

第二刀：跨组件轨迹重建。
一次智能体运行会跨过模型、harness、沙箱这些独立管理的组件，重叠、失败、重试都有。组件本地日志和时间戳不够用来归因。Aries 在模型和工具边界传播轨迹与事件标识符，用统一事件模式记录因果和顺序，再把组件指标挂到这些事件上。没有这一刀，就只能说“这次任务慢”，说不清是解码慢、harness 编排慢，还是某个工具沙箱堵了。

第三刀：有状态工具执行的可移植观测。
工具作用在持久环境上，环境状态会影响后续轨迹；不同沙箱基底的生命周期和遥测接口又不一样。Aries 对环境设置、工具调用、状态连续性、结果收集和遥测挂钩做标准化，由适配器映射到具体基底。工具执行因此成为轨迹里一等公民的阶段，而不是“一次 RPC 返回了什么”。

这三刀合在一起，Aries 更像智能体系统研究的 实验操作系统：先保证任务没被偷偷换掉，再保证事件能串成因果链，最后保证工具阶段在不同基底上可比。第 4 节的四条结论，都建立在这个前提上。若没有 Aries，生产轨迹只能给出分钟级、匿名化的宏观画像，开源实验也无法把延迟拆到 harness / 工具 / 模型。

3. 四条结论分别在否定什么

第 4 节的标题是直白的：Why Is Modern Cloud AI Infrastructure a Poor Fit for Agents? 方法分两段：先看一条八小时工作日的生产轨迹（十个 LLM 服务实例的请求长度，100 个智能体沙箱的每分钟 CPU/内存），建立规模感；因为运维轨迹没有同步的跨层事件，再用 OpenHands、Hermes、OpenClaw，在 SWE-Bench Pro、Terminal-Bench 2、DeepResearch Bench 上各抽 20 个任务、重复五次，用 Aries 做受控拆解。默认模型是本地 SGLang 上的 Qwen3.6-35B-A3B-FP8，单机 96 核 + 一块 94 GB HBM 的 H100。

下面四条 Takeaway，最好读成对现有四个隐含假设的否定，而不是四条互不相关的经验数字。

3.1 否定“把模型调快，任务就快”

图 3 显示工具执行可占总延迟的 13%–48%，Hermes 在 Terminal-Bench 2 上到 48%；harness 编排还可再占端到端时间的最多 9%。图 4 更关键：LLM 延迟固然重要，工具调用却有跨越多个数量级的长尾。智能体在工具返回之前无法前进一步，所以这段主机侧时间不是“应用开销”，而是关键路径。

Takeaway 1 因此不是“工具也挺慢的”，而是：端到端任务时长在很大程度上取决于 harness 与工具沙箱，而不仅仅是模型推理。 继续把优化预算全部砸进 GPU 内核与 KV 缓存，会优化一个越来越不主导任务完成时间的子系统。

读这条时要注意口径：逐步计时在商业集群里过敏、拿不到，这条拆解来自开源数据集与 harness。生产轨迹只提供了“智能体上下文比对话请求大得多”这类宏观证据，并没有直接给出 48% 这个比例。把开源拆解外推到生产是合理假设，但不是同一张图上的同一组数。

3.2 否定“上下文越长越好，KV 留下即可”

传统 LLM 服务里，上下文寿命通常等于请求–响应循环，查询结束后可以回收 KV 页。长时程智能体则生成单调增长的历史，并贯穿整条轨迹。生产侧（图 5）已经看到：智能体每请求 KV 占用显著大于普通对话；在 Qwen3.6、固定 25 GB 预算下，最大驻留容量从超过 176 个并发请求掉到 40 个，约 4.4×。

受控扫描（图 6）把“更长是否更准”拆开了，这是本节最有信息量的结果：

  • OpenClaw 在 DeepResearch Bench 上于 160K 达到 95% 峰值成功率，这一点恰好对应上下文溢出被消除。
  • Terminal-Bench 2 与 SWE-Bench Pro 在溢出压力消失后趋于平缓甚至下降，最佳窗口是中小上下文，不是最大窗口。

也就是说，存在任务相关的上下文充分性阈值。超过阈值后，多留的历史主要在锁 GPU Token 池，而不是在买精度。32 个并发、256K 上下文的压力测试进一步显示：积压存在，但 Token 池利用率冲到 97%，引擎中位数只能维持 22 个并发，调度器开始抢占，p95 内部排队 7.9 秒。

Takeaway 2 的系统含义是：上下文管理不能再是引擎里被动的“能留就留、满了再压”。它必须变成轨迹感知的控制面决策——估计这段历史对后续步骤的效用，决定压缩、分层放置还是迁移。论文在第 5.2 节把处方写出来了，但 没有给出如何估计“未来效用”的方法。那才是把这条结论做成系统时最硬的问题。

另一个容易被忽略的点：他们为了做精度研究，把 OpenClaw 的任务成功率标定在 50% 附近再采样。这能让上下文扫描看得到上升和平台，但也意味着结论对“已经很强、很少溢出”的模型/harness 组合，幅度会变小。

3.3 否定“有状态缩至零已经能回收沙箱空闲”

智能体把 CPU 处理引进了关键路径：模型推理时沙箱往往空转，工具一来又出现短促的 CPU/内存尖峰。静态按峰值供给会在大部分时间里把资源闲置在那里。AWS Lambda MicroVM、Google Agent Sandbox 等用快照捕获/恢复（C/R）做有状态缩至零，看起来正好对上这种空闲–突发模式。论文要证明的是：模式对上了，经济账没对上。

生产侧每分钟采样已经呈尖峰；开源侧每秒采样把形状画得更清楚：轨迹的大部分时间里 CPU 空闲超过 80%、内存空闲超过 50%；一旦条件限制在“正在进行工具调用”，空闲比例都降到 20% 以下。沙箱不是低利用率的常驻服务，是 大部分时间闲着、一工作就突发 的执行器。

然后他们对记录的工具调用轨迹，按 AWS Lambda MicroVM 计费模型做解析式重放，扫描不同保活超时：

  • 每次工具调用后拆掉沙箱：实例成本可降 65%，但反复快照 C/R 把客户总成本推到持久基线的 4.9×。
  • 保活更久：C/R 下降，空闲时间上升。
  • 理想系统（能回收空闲、又不付今天这种 C/R 税）相对持久沙箱，总成本有约 3× 的下降空间。

Takeaway 3 因此是一个定价与机制的指控，而不只是利用率画像：用户和云厂商都有缩短保活的动机，但当前快照机制把人逼进“为闲置付费 vs 为反复快照付费”的假选择。弹性是必要的，今天的 C/R 不是那条弹性该走的路。

读这条时必须记住：3× 和 4.9× 来自计费模型上的解析重放，不是他们真的实现了一个理想弹性系统再测出来的。DeepResearch 因工具 CPU 低被排除，说明结论对“重执行型工具”（终端、代码、浏览器）更强，对“轻查询型工具”更弱。

3.4 否定“沿用现有不可信代码沙箱就够了”

第 4.4 节的论证链和其他三节不同。前三节是 Aries 测出来的；这一节主要用外部证据做代理：Linux 内核 CVE 中明确写了用 AI 工具发现/修复的比例，2026 年 Q2 相对历史纪录约 3.5×，并引用“智能体花几美元就能写出利用”。前沿模型已在内核、VMM 里找到高级漏洞，于是推断智能体具备突破云隔离的可能性。

Takeaway 4 本身作为方向是对的：工具沙箱应缩小攻击面，并按任务裁剪权限（深度研究不必给 bash，给客户机内沙箱化的 JS 运行时即可），长期甚至要走向 seL4 这类形式化验证。但作为论文里的测量结论，这一节是最弱的一环：

  • “提交信息里提到 AI 工具”≠“智能体在云租户环境里自主攻破隔离”。
  • 补丁审查系统 Sashiko 推高 CVE 发现，说明的是防御侧/审计侧能力，不能直接等于进攻性租户风险。
  • 没有 Aries 轨迹级的攻击面测量，也没有在 OpenClaw / Hermes 的工具权限集上做消融。

把它读成 愿景动机 比读成 与前三条同等级的实证 更合适。第 5.4 节的处方（主机 VM 隔离 + 客户机内核沙箱、按任务最小权限、减少侧信道暴露）仍然值得做，只是证据类型不同。

4. 愿景层：诊断开出的四张处方

第 5 节不再报数，而是把四条否定翻译成智能体原生服务系统的四个协同方向。图 12 把它们画在 Runtime / Control / Execution 三层上。

轨迹捕获指标。
不要只用 Token 吞吐。图 13 表明：Token 吞吐可以和“每小时解决的任务数”“每百万生成 Token 解决的任务数”背离——模型在忙，任务不一定在推进。终点指标（任务成功率、有效吞吐）要等轨迹结束才有，不能指导在线决策。真正缺的是从轨迹事件里推导 在线效用信号：这一次模型调用、这一次 harness 决策、这一次工具交互，到底贡献了多少增量进度，花了多少时间和资源，是有效动作还是原地打转。

轨迹感知的上下文控制面。
监视任务状态与内存状态，决定上下文保留、压缩还是分层放置。当预期精度收益不再抵偿 KV 成本时请求压缩；当释放的加速器容量超过迁移开销时，把长生命周期状态迁到更便宜的层级。这是 Takeaway 2 的系统化。

智能体感知的弹性控制面。
主机侧需求高度间歇，水平自动扩缩在这个粒度上太贵。需要的是 垂直的、模型感知的 单沙箱扩缩：用模型/harness/运行时信号预测即将到来的核数与内存尖峰。同时必须把执行状态从物理放置里解耦——哪些状态必须驻留，哪些可以外置或重建，挂起、迁移、改大小时如何保持一致。这是 Takeaway 3 的系统化，而且明确拒绝“再做一次传统 HPA”。

攻击面最小的沙箱。
按任务裁剪工具、网络、文件系统；主机侧 VM 隔离与客户机侧内核沙箱叠在一起；长期用形式化验证面对越来越强的进攻性模型。这是 Takeaway 4 的系统化。

把第 5 节读成论文的“未来工作”会低估它。这篇论文的贡献分配其实是：Aries 让问题可测，第 4 节证明旧抽象失效，第 5 节给出新抽象应该长什么样。 它故意不在本文实现那套控制面。这是定位，也是边界——读者若期待一套可部署的 Agent OS，会觉得后半段偏宣言。

5. 实验与论证该打的折扣

这篇论文的说服力主要来自“生产画像 + 开源受控拆解”的双轨，而不是来自某个 2.9× 的加速比。双轨有 complementary 的力量，也有对不上的缝：

  1. 生产轨迹没有同步跨层事件。 八小时、十个实例、一百个沙箱的分钟级采样，足以说“智能体 KV 更大、沙箱更尖峰”，不足以把延迟拆到 harness 或某类工具。48%、9%、7.9 秒这些细数，全部来自开源复现。
  2. 开源样本不小但也不大。 三套基准各 20 题、五次重复，且为精度研究把 OpenClaw 调到约 50% 成功率。对“上下文阈值随工作负载而变”这种定性结论是够的；对给出普适百分比则偏紧。
  3. 单机单卡。 96 核 + 一块 H100 能把 KV 池打满、把排队打出来，但看不到多租户集群里的干扰、跨节点沙箱、网络存储。弹性控制面若要在生产成立，还缺这一层。
  4. 沙箱成本是计费模型重放。 4.9× 与 3× 依赖 AWS Lambda MicroVM 的价格结构。换 Google / 自建快照栈，绝对值会变；“C/R 税抵消了缩至零收益”这个方向更可能稳住。
  5. 安全节是代理指标。 见 3.4。
  6. 正文有整理痕迹。 英文稿里出现 “Fig. and Fig.” 这类未填完的引用，个别图注和 HTML 粘贴残留。不影响主结论，但说明测量论文的文本完成度不如其问题意识。

这些折扣不推翻主线。它们说明：Aries 已经能把“旧基础设施不适合智能体”说到可辩护，但还不能把第 5 节的控制面设计参数标定出来。

6. 和 SpecBox 怎么对齐

同目录下的 SpecBox 正好是这篇诊断的一种运行时处方，两篇不宜混成一篇，但对着读很有帮助。

Aries SpecBox
体裁 测量框架 + 问题重构 预测性服务运行时
基本单元 轨迹 仍是逐步的 ReAct 循环，但把环境生命周期当成调度对象
对沙箱的判断 大部分时间空闲、工作则突发；快照缩至零不划算 不要永久预留，也不要每次从零等；用推测把准备藏进解码/执行
对指标的判断 Token 吞吐会与任务有效吞吐背离 几乎不讨论任务成功率，优化的是 P99 与内存
对上下文的判断 超过充分性阈值只伤容量 基本不碰 KV / 上下文控制面
实现位置 实验与遥测 引擎与 MCP 沙盒之间的中间件

Aries 的 Takeaway 3 解释了为什么 SpecBox 那种“只准备、不常驻、不靠反复 C/R”的路值得走：永久预留浪费，激进挂起又太贵。SpecBox 则没有回答 Aries 的另外两半——轨迹级指标、上下文控制面、以及按任务收缩攻击面。把两篇合在一起，智能体云基础设施缺的其实是三块控制面：上下文、沙箱弹性、安全策略；SpecBox 只在弹性这一块里做了“预测性预热”这一种机制。

7. 对工程实践真正可搬走的东西

不必实现完整 Aries，下面几条已经能改现有平台的优先级：

  1. 把一次用户请求记成轨迹，而不是记成多次 LLM API 调用。 没有共享的轨迹 ID，就无法回答“这次任务慢在哪”。这是可观测性改造，不是模型改造。
  2. 在端到端延迟里单独列出 harness 与工具。 若工具已经接近或超过模型时间，下一季度的优化对象就不应再只是引擎。
  3. 不要默认“最大上下文 = 最好”。 按任务类型扫一遍充分性阈值。深度研究类可能需要去溢出；终端/修代码类可能在中等窗口更好。超过阈值的历史应被压缩或下沉，而不是继续占 HBM。
  4. 沙箱按峰值静态供给，和每次调用都做快照，都可能是错的。 先画自己的空闲–突发曲线，再决定保活。若空闲超过 80%，钱不该花在更大的常驻池，而该花在更便宜的状态外置/预测性准备上。
  5. 工具权限按任务裁剪，而不是给一个万能 bash 沙箱。 即便不谈形式化验证，深度研究、浏览器、代码修改对攻击面的需求也完全不同。
  6. 评测任务与执行栈解耦。 换模型或换沙箱时如果任务定义跟着变，对比没有意义。哪怕只在内部把“成功语义”写成独立配置，也比绑死在某个 harness 里强。

8. 局限，以及读完之后还该问的问题

Aries 把智能体服务从“更快的 generate”重新定义为“可持续的轨迹”，并且用生产与开源两条线把失效模式钉住。它留下的缺口，和它打开的问题一样明确：

  • 在线效用信号怎么造？ 第 5.1 节指出 Token 吞吐会与任务进度背离，但没有给出可计算的增量进度估计。没有这个信号，轨迹感知调度只是事后分析，不能闭环。
  • 上下文未来效用怎么估？ 充分性阈值是离线扫出来的。生产控制面需要在轨迹进行中做保守估计，估错就会伤成功率。这是系统问题，也是模型问题。
  • 垂直弹性的预测信号是什么？ 论文问了，没有答。是下一个工具的类型、参数大小、历史 CPU 曲线，还是模型自己的计划文本？SpecBox 用的是工具转移图与流式意图，恰好是这个方向上的一个具体答案，但覆盖的是启动时机，不是核数/内存的垂直扩缩。
  • 状态与放置如何解耦？ 文件、进程、缓存、网络连接，哪些必须随沙箱迁，哪些可以重建，决定了弹性的真实成本。Aries 只证明今天的整卷快照太贵。
  • 安全论证需要轨迹级实验。 最小权限、双层隔离、侧信道，都还停留在方向。要对 OpenClaw 一类真实工具集做攻击面消融，才配得上前三节的测量标准。

如果只用一个判断收束：Aries 证明了智能体云基础设施的危机不在某一层的性能数字，而在抽象用错了——用请求级指标和服务原语，去支撑轨迹级、有状态、带工具的工作负载。 它给出的不是可上线的控制面，而是一张必须改抽象的地图。对正在把 OpenClaw / Claude / Codex 类智能体接到现有 LLM 集群上的人来说，这篇论文最有用的不是 48% 或 4.4× 这些数字本身，而是它强迫你改问的三个问题：任务有没有在推进、这段上下文还值不值得占着 GPU、这个沙箱此刻到底该驻留、挂起还是按任务缩小权限。

3 - [论文]SpecBox：用于高效LLM智能体服务的推测性沙盒调度方案

[论文]SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

3.1 - 论文介绍

论文信息和介绍

基本信息

发表时间： 2026年8月5日

论文PDF： https://arxiv.org/pdf/2607.29069

论文 HTML 页面： https://arxiv.org/html/2607.29069v1

作者信息

3.2 - 论文英文版本

论文英文版

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

Abstract

As LLM agents increasingly rely on the Model Context Protocol (MCP) to invoke isolated external sandboxes, disaggregated sandbox deployment introduces a fundamental tension between resource utilization and interactive tail latency. Persistent long-lived sandbox reservations incur excessive memory overhead at scale, while lazy on-demand instantiation generates severe cold-start penalties that degrade response performance under multi-tenant, multi-turn agent workloads. To resolve this dilemma, we present SpecBox, a runtime built around speculative sandbox preallocation tailored for dynamic LLM agent execution pipelines.

At its core, SpecBox implements keyword matching and streaming semantic embedding to enable intent-driven sandbox prewarming, which identifies pending tool execution demands mid-LLM token generation and fully overlaps sandbox bootstrapping with model inference. To extend prewarming windows across sequential agent steps, the framework leverages context-aware stochastic prefetching atop a sandbox dependency graph to probabilistically forecast future sandbox switches ahead of execution. We complement these speculative mechanisms with two orthogonal optimizations: a semantic result cache that prunes redundant repeated sandbox invocations, and a dedicated out-of-band shared-memory transport plane that bypasses conventional network serialization to deliver zero-copy artifact transfers. Evaluated on high-concurrency multi-turn agent traces, our prototype demonstrates that SpecBox cuts P99 end-to-end latency by up to 2.9× relative to the on-demand sandbox baseline, while slashing peak memory consumption by 45.9% compared to permanently reserved sandbox deployments.

Keywords: LLM Agent, Execution Runtime, Predictive Prewarm

1 Introduction

Modern LLM agents are evolving from conventional text generators into autonomous systems that iteratively reason, plan, and execute external actions (Yao et al., 2022; Li et al., 2024b; Wang et al., 2024). Unlike traditional LLM serving workloads that primarily optimize token generation (Agrawal et al., 2024; Zhang et al., 2025), agent execution forms a stateful run-loop where an LLM-based controller repeatedly invokes external tools (Microsoft, 2026b; GitHub, 2026; Datalayer, 2026; Neo4j, 2026) such as code execution, web automation, and data processing services. For security, isolation, and reproducibility, these tools are increasingly deployed as independent sandbox environments and accessed through standardized interfaces such as the Model Context Protocol (MCP) (Anthropic PBC, 2024). Consequently, the latency of agent execution is no longer determined solely by model inference, but also by the efficiency of coordinating heterogeneous external execution environments.

Modern cloud-native infrastructures increasingly adopt serverless sandbox execution to support large-scale concurrent agent sessions (Amazon Web Services, 2026; Google Cloud, 2026; Microsoft, 2026a), i.e., the execution environments are initialized on demand rather than being reserved as long-lived instances. This design enables the required elasticity for multi-tenant agent workloads but introduces non-negligible latency (Puliafito et al., 2022; Yu et al., 2024; Xu et al., 2024; Stojkovic et al., 2023a). Each tool invocation may incur sandbox initialization overhead — such as image loading, filesystem preparation, namespace configuration, and runtime handshake — introducing a second-level delay before execution begins (Huang et al., 2026). In multi-turn agent workflows, such startup overheads accumulate across successive tool invocations, significantly degrading end-to-end responsiveness and limiting the practicality of interactive agent services.

Refer to caption Figure 1.Proactive and overlapped execution v.s. the native vanilla approaches

The main cause of this latency is not sandbox initialization, but the sequential execution model used by the vanilla implementation of the de facto agent runtimes (e.g., AgentScope(Gao et al., 2024), AutoGen(Wu et al., 2023), and LangGraph (Langchain-ai, 2026)). These systems use a reactive and sequential model without pipelined orchestration: the stage of sandbox preparation begins only after the stage of LLM inference finishes the token generation, and the tool invocation is fully determined. As a result, the preparation of the environment remains entirely exposed on the critical path, leaving idle periods between model reasoning and tool execution. While GPUs are occupied generating tokens, CPU-side resources remain underutilized — once sandbox initialization begins, the GPU has to wait for external execution to complete. As illustrated in Fig. 1, the reactive model serializes LLM reasoning and environment preparation, without effectively overlapping computation with environment provisioning.

Existing runtime optimizations such as speculative execution (Stojkovic et al., 2023b; Li et al., 2024a), prewarming (Mahgoub et al., 2022; Sui et al., 2024), workflow orchestration (Liu et al., 2023; Li et al., 2023), and caching (Xu et al., 2024; Lu et al., 2024) are well studied in serverless and cloud systems. However, they do not directly transfer to LLM agent runtimes because agent execution differs fundamentally from conventional request processing. Prior work usually assumes fixed execution targets or pre-defined workflow graph before runtime. In contrast, autonomous agent workflows are produced online via auto-regressive reasoning: tool calls emerge progressively from streaming token generation and remain uncertain until enough semantic context is available. Multi-turn sessions further evolve based on intermediate observations and outcomes, making future tool choices both workflow-dependent and highly dynamic. These characteristics make simple extensions of existing techniques infeasible.

In the context of LLM agent runtimes, the key idea is therefore to pre-launch the most likely sandbox before it is requested, overlapping environment preparation with ongoing LLM generation. However, realizing this idea for autonomous agent systems introduces three unique challenges. First, within a single step (agent iteration), intents must be inferred from early token streams with only partial semantics; using short contexts with low thresholds or broad candidate sets improves overlap but risks false-positive prewarming and wasted memory/CPU. Second, across steps, predicting which tool will be invoked in future steps becomes increasingly unreliable as the prediction window extends; prewarming all plausible successors would revert to reserved-deployment costs. Third, prewarming alone removes only sandbox startup delay: repeated tool executions and large artifact transfers can still remain on the critical path even when a sandbox is ready. The system must therefore reuse semantically equivalent execution results and decouple bulk data transfer from control signaling, while preserving protocol compatibility and user-visible semantics.

This paper presents SpecBox, a predictive serving system that orchestrates tasks of an agent workflow efficiently. SpecBox overlaps LLM execution with environment preparation, breaking the rigid sequential dependencies in vanilla implementation of agent runtimes. SpecBox diminish end-to-end latency through three synergetic techniques: i) intent-aware sandbox prewarming that speculates execution intents on the basis of streaming token outputs, at a proper time, and overlap sandbox preparation with ongoing generation within a step; ii) stochastic sandbox prefetching, leveraging historical agent execution traces to anticipate future sandbox needs and prepares the sandboxes for the next step during the current step, thereby lowering cold-start latency; and iii) reuse-aware data transmission that exploits semantic similarity to bypass redundant tool execution through semantic caching and decouples large artifact delivery from control signaling through an out-of-band data path, eliminating unnecessary computation and serialization overhead. SpecBox’s prototype is implemented and integrated with the open-source AgentScope framework (Gao et al., 2024). SpecBox is framework-agnostic and highly extensible to any LLM agent serving infrastructures. On diverse multi-turn, high-concurrency agent benchmarks, SpecBox outperforms state-of-the-art serverless runtimes while preserving workflow correctness and protocol compatibility. SpecBox reduces P99 latency by up to 2.9× and peak host memory usage by 45.9%. This paper makes the following key contributions.

  • characterizing the latency-critical path of multi-turn LLM agent execution and proposing a new sandbox prewarming approach (C1) via inferring streaming token-level intents such that sandbox preparation can be better overlapped with the ongoing LLM inference (§ 3.1).

  • devising stochastic sandbox prefetching mechanism (C2) to reduce cross-step cold-start latency by mining historical agent execution traces and pre-warming likely sandbox environments over a sandbox dependency graph (§ 3.2).

  • introducing reuse-aware data transmission (C3), via out-of-band transport and semantic caching, efficiently transferring and reusing intermediate execution artifacts while eliminating redundant sandbox initialization (§ 3.3).

2 Background and Motivation

2.1 Agent Workflow and Environment

Large Language Model (LLM) applications have shifted from monolithic, single-turn chatbots to distributed autonomous agents. Traditional workflows rely on manually predefined execution graphs, where the operation sequence is fixed before runtime. In contrast, modern agent workflows integrate autonomous decision-making: given a high-level objective, an agent can dynamically decompose tasks, select execution sandboxes or external environments, and adapt its trajectory based on intermediate observations. As a result, the workflow is no longer statically specified but instead emerges from continuous interactions between LLM reasoning and external environments (Yao et al., 2022; Li et al., 2024b; Wang et al., 2024).

An agent workflow is realized through a sequence of event-driven execution steps, where each step represents an intermediate task that involves reasoning, planning, or invoking local and remote sandboxes. Tasks that require file system interaction, code execution, web access, or queries against proprietary databases are delegated to isolated external execution layers (environment or tool sandboxes). Standardized protocols—most notably the Model Context Protocol (MCP) (Anthropic PBC, 2024) proposed by Anthropic—formalize this interface boundary. MCP specifies a common communication layer over transports such as HTTP and Server-Sent Events (SSE), thereby transforming tool invocation into a set of loosely coupled microservices. This architecture is closely aligned with emerging agent-oriented operating systems that conceptualize LLMs as central processing units and external environments as peripheral devices (Mei et al.,; Gao et al., 2024). From a systems-engineering standpoint, an agent’s execution increasingly manifests as a continuous stream of heterogeneous, RPC-like interactions between the LLM inference core and multiple environment services, rather than as a monolithic, localized compute workload.

2.2 Serving Runtimes

Disaggregating Agent Engines from their underlying execution environments enables substantial scalability and architectural flexibility. However, deploying these decoupled components within cloud-native, multi-tenant cluster infrastructures introduces significant challenges related to performance and resource management. Service providers must carefully balance isolation guarantees, resource efficiency, and end-to-end latency, thereby exposing an inherent trade-off between long-running (reserved) and serverless (on-demand) serving paradigms.

Reserved Agent Runtime. A straightforward strategy (Tan et al., 2025; Huang et al., 2026) is to maintain pre-initialized, always-on containers for each user tenant and its associated sandboxes. However, in contemporary agent ecosystems comprising thousands of fine-grained tools, sustaining all execution environments in a warmed state becomes prohibitively resource-intensive: idle containers incur substantial host memory and CPU overheads (Xu et al., 2024; Stojkovic et al., 2023a), which in turn lead to pronounced cluster underutilization and resource interference, thereby rendering large-scale multi-tenant deployments economically unsustainable.

On-demand Agent Runtime. To enhance resource utilization, modern cloud-native platforms commonly employ on-demand runtime provisioning (Du et al., 2020; Shahrad et al., 2020), wherein tool sandboxes are instantiated (“cold-started”) only upon explicit requests from agents. However, this design choice introduces substantial tail latency: the on-demand initialization of an isolated container incurs a series of serialized overheads, including container image download and extraction, network namespace setup, virtual file system mounting, and application-level handshake procedures. Under bursty workloads or in complex multi-turn workflows, these multi-second cold-start delays accumulate along the agent’s end-to-end critical path, thereby inflating P99 tail latency and degrading the interactive quality of service (QoS) of real-time intelligent agents.

2.3 Execution Bottlenecks in Agent Runtimes

In contrast to conventional LLM serving, an agent session constitutes a continuous, stateful execution loop composed of multiple iterative reasoning and sandboxed execution steps. Prevailing agent frameworks (Gao et al., 2024; Wu et al., 2023; Langchain-ai, 2026) employ a reactive execution paradigm, in which each tool invocation is initiated only after the LLM has completed its reasoning step. This design induces a strictly serialized dependency chain between model inference and environment interaction. The resulting execution workflow is depicted in Fig. 1.

As illustrated in Fig. 2(a), the end-to-end latency of a single execution step N can be broken down into several parts:

The first two components are associated with LLM inference, including context processing and token generation. The remaining three define the optimization boundary for an agent runtime: Tenv_prep(N) is the time duration to get the selected sandbox environment ready, Tdata_io(N) is the time to exchange invocation inputs and results, and Tsandbox_exec(N) is the time spent executing the tool.

SpecBox targets these time frames without changing the LLM inference stack by overlapping preparation with reasoning, reducing data movement, and eliminating redundant executions. We derived the following observations that motivate this study.

Observation #1: In-step preparation can overlap with token generation. In today’s reactive execution model, environment initialization cannot start until the LLM emits a explicit sandbox invocation, even though the prompt, plan, and partial token stream often already reveal the likely sandbox. Using this partial evidence to start preparation earlier can put the prewarm even forward, ahead of the corresponding invocation. This interval lets sandbox startup, runtime connection setup, and other preparation work overlap with the remaining Tgeneration(N), rather than placing all of Tenv_prep(N) on the critical path. As shown in Fig. 2(b), sandbox initialization requires several seconds and can approach 20 seconds for heavyweight sandboxes, rendering this overlap mechanism critical for minimizing the effective preparation latency observed by the system.

Observation #2: Cross-step execution can advance the prewarming. Consecutive steps within an agent session frequently exhibit strong temporal locality. For instance, in a large language model (LLM) serving workload, a paper search step is often followed by a document reading step that processes a retrieved document, and a data analysis step is commonly followed by a figure generation step. Nonetheless, prevailing runtime systems generally treat each invocation as an independent event, discarding workflow context once a step is initiated. In contrast, the runtime can exploit the execution window of the current sandbox to proactively instantiate and prewarm the most likely subsequent execution environments before step N+1 begins, thereby making the intra-step prewarm more in-advance beyond the current decoding stage.

Observation #3: Redundant execution and excessive unnecessary data transfer. In multi-turn agent workflows, later reasoning steps often revisit information that was already produced, such as querying the same document again or re-running deterministic analyses. However, conventional runtimes still re-execute these operations even when an equivalent result is already available, repeatedly paying the cost of Tsandbox_exec(N). Meanwhile, sandbox interfaces often couple the synchronous transfer of large intermediate artifacts (e.g., files, images, structured outputs) with control messages, causing Tdata_io(N) to scale with artifact size. Intuitively, one can accelerate the agent execution by caching and reusing prior execution results and decoupling artifact transmission from control signaling, thereby eliminating redundant work and data transfer overhead.

These observations reveal optimization opportunities at three points in the agent execution loop: overlapping the environment preparation with the LLM operations within a step, advancing the environment preparation of the next step in the current step, and eliminating repeated execution or data movement. Existing LLM serving frameworks primarily focus on optimizing model execution — such as prefill and decoding phases, as well as KV cache management — while serverless runtimes primarily reduce the overhead associated with container initialization without considering user-wise intent. However, neither of them directly addresses how an agent-oriented runtime can leverage these optimizations while maintaining resource efficiency and preserving the intended semantics of tool invocation and usage.

Figure 2.Execution bottlenecks in agent runtimes. (a) Execution-step latency breakdown. (b) Mean cold-start latency across 32 sandboxed environments, including all MCPBench (Wang et al., 2025) sandboxes and additional commonly used environments. Most sandboxes initialize in 2–4 seconds, while resource-intensive environments can require up to approximately 20 seconds.

Refer to captionFigure 3.Intent-aware sandbox prewarming: the plan agent expands user requests into ReAct steps, and keyword plus semantic predictions jointly decide to prewarm the sandbox or not.

Refer to captionFigure 4.Tradeoffs among routing policies. Keyword routing trades early decisions for potential false positives; semantic routing and intersection delay a precise decision; union assembly retains precision while allowing either router to trigger prewarming early.

2.4 Research Challenges

Designing an effective execution runtime to accelerate the LLM agent serving is faced with the following challenges.

Challenge #1: Early intent prediction vs. resource waste. Intent speculation of tool use from given incomplete token stream is critical to ensure the timeliness of sandbox environment preparation whilst avoiding unwanted tool launch due to inaccurate intent prediction. Early warmup must rely on highly ambiguous early-stage tokens (e.g., general-purpose verbs such as read and get reused across different MCP tools), which often causes false-positive sandbox activation, wasting resources and potentially triggering host-level OOM in multi-tenant deployments.

Challenge #2: Cross-step in-advance prewarming vs. non-deterministic workflow execution. It is advantageous to select the subsequent-step sandbox environments in advance of the manifestation of the next intent. The agent workflow executes in a probabilistic manner, admitting multiple plausible successor states. Prewarming all potential successors incurs a computational and resource overhead comparable to that of a fully reserved deployment, whereas prewarming only a single successor substantially reduces the opportunity to amortize sandbox initialization costs and yields only limited latency improvements. Instead, the runtime system should exploit historical transition data to rank a small set of highly probable target environments, then refine this choice using the signal available at the current step. This balances resource utilization against latency reduction while preserving the ability to conceal sandbox initialization behind ongoing computation.

Challenge #3: Reuse and data-path efficiency vs. sandbox compatibility. We must skip redundant work without changing what the agent observes from a sandbox invocation. Exact reuse is safe but misses semantically equivalent requests expressed differently; unconstrained semantic reuse may yield incompatible results. Similarly, putting large artifacts in normal RPC messages preserves compatibility but keeps data movement proportional to payload size, while replacing the control interface would break existing tools. Hence, the runtime must detect compatible reuse, reject unsafe approximations, and decouple bulky artifacts from control signaling while preserving sandbox-execution semantics.

3 Design of SpecBox

We focus on three critical time periods within the execution of reactive agents: environment preparation, data movement, and sandbox execution. This section how SpecBox accelerate them with decoupled yet inter-connected optimizations: intent-aware sandbox prewarming that infers an in-step tool intent early to overlap the environment preparation with the decoding of LLM serving (§ 3.1); stochastic sandbox prefetching that exploits cross-step workflow regularity to further put the preparation forward (§ 3.2); and reuse-aware data transmission that avoids redundant execution and removes large artifacts from the control path (§ 3.3).

Refer to caption

Figure 5.Stochastic sandbox prefetching in SpecBox. Execution history predicts likely sandboxes for subsequent ReAct steps.

3.1 Intent-Aware Sandbox Prewarming

Naive reactive runtimes await a full tool invocation before sandbox preparation, even if prompts, evolving plan, and partial generation already reveal user intents. SpecBox instead prewarms the sandbox earlier by inferring streaming token-level intents, better overlapping sandbox preparation with ongoing LLM inference.

SpecBox makes the best use of the LLM docoding period Tgeneration(N) to obtain just enough user intents for launching accurate sandboxes without compromising the timeliness of prewarming. This is done by streaming context to two independent routers. As shown in Fig. 4, there exists a dilemma: earlier intent predictions overlap more with the LLM decoding period but are less reliable and can waste sandbox capacity. Keyword Router acts early but must balance false positives against a stricter threshold; Semantic Router is more precise but slower. Requiring both predictions incurs the Semantic Router’s delay, whereas opportunistically combining their outputs preserves high precision while allowing earlier decisions. Hence, SpecBox proposes an intent-aware sandbox prewarming mechanism that combines the two asynchronous intent predictions, rather than considering either one on its own to be sufficient.

Keyword Router. The Keyword Router scans the stream against tool-specific keyword profiles and can emit a candidate within microseconds of a distinctive token, preparing the environment while the LLM continues decoding. Common terms like research, search, or slide occur in many tool descriptions: triggering on a single match creates a large prewarm set with many false positives, while requiring many matches will delay the preparation, missing good chances of overlapping. We therefore apply a threshold γ on the number of matched tool-specific keywords. In Fig. 3, surface cues in the user request and evolving plan let the Keyword Router emit likely sandbox candidates before the plan agent finishes the current ReAct step. The threshold balances early activation with resource waste; the chosen configuration is given in § 5.3.

Semantic Router. In parallel, the Semantic Router compares the active context with tool-intent representations. It captures requests whose wording overlaps little with a tool profile and disambiguates generic keyword cues using the plan and prior generation. In Fig. 3, it can identify candidate sandboxes from the broader task intent even when no single token uniquely identifies a tool. Its tradeoff is temporal: reliable semantic evidence typically needs a longer prefix, so a semantic-only decision often arrives too late to hide the cold start latency of a heavy-weight sandbox such as PaperSearch (OpenAGS, 2026), and Neo4j (Neo4j, 2026). The Semantic Router is an asynchronous, complementary source of candidates that can recover intents the Keyword Router misses.

Union Assembly. Let 𝒮key(N) and 𝒮semantic(N) be the candidate sets from the two routers at token step N. The lower-left quadrant of Fig. 4 shows why prewarming only their intersection is unsuitable: it boosts apparent precision but makes every trigger wait for the Semantic Router and drops valid tools whenever either router has imperfect recall. Instead, we use the lower-right policy:

Each router manages its own false positives, and the union lets the first credible signal start preparation. Fig. 3 shows the resulting behavior: routers independently generate candidates from the request and partial plan, then form a unified prewarm set for the sandbox manager. Explicit intents benefit from the early prewarm of the Keyword Router, while implicit intents are handled by the Semantic Router. As will be shown in § 5.3, asynchronous union can minimize waiting time and eliminate cold starts in the evaluated routing workload.

Refer to caption

Figure 6.Stochastic sandbox prefetching using a first-order Markov model. Left: SDG-based Markov state transition graph with edge probabilities from observed counts. Right: example session showing thresholded, budgeted prefetching across turns, where predicted successors are prewarmed before the next step commits.

3.2 Stochastic Sandbox Prefetching

Intent-aware sandbox prewarming can only use the remaining decoding time of the current step, which is inadequate when a non-resident sandbox takes longer to start than the marginal token-generation window. In reality, steps in an agent workflow are non-deterministic but generally follow a probabilistic model. For instance, after a paper search, an agent may read a returned document; after data analysis, it may generate a figure or report. SpecBox exploits these cross-step probabilistic patterns to navigate the environment preparation in advance during Tsandbox_exec(N), before the next step commits to a specific invocation.

Stochastic Markov Process Modeling. Let 𝒱 denote the sandbox state space, where each state is a sandbox type or a typed tool-state tuple in the state-level SDG. For an execution trace {S(1),…,S(N)}, SpecBox records step-to-step transitions in a directed sandbox dependency graph (SDG). For each ordered pair (vi,vj), we maintain transition counts:

The next-state probability is estimated by a first-order Markov model with Laplace Smoothing (Laplace, 1812):

where α=1 avoids zero-probability collapse for sparsely observed transitions.

Prefetching Under Budget Constraints. Given current state vi, SpecBox first identifies sandbox candidates with non-trivial cold-start cost:

where Lj denotes the estimated cold-start penalty of sandbox vj and λ is a lightweight cost threshold.

SpecBox then filters out low-confidence successors and retains only high-probability candidates:

where τ controls false-positive prewarming. Finally, SpecBox selects the top-B sandboxes under a fixed budget:

where 𝒞i is the filtered candidate set, 𝒜i is the final budgeted prefetch set, and B bounds the number of sandboxes prewarmed per step. This policy is intentionally lightweight and can be executed outside the LLM generation critical path.

Online Update. After each completed sandbox invocation, SpecBox appends one transition edge to SDG and updates Ci,j and Pi,j asynchronously in the background prefetch worker. Thus, the predictor continuously adapts to evolving multi-turn workflows without blocking foreground agent execution. Fig. 5 shows how stochastic sandbox prefetching operates together with intent-aware sandbox prewarming in a complete multi-step workflow. After step N finishes in sandbox vi, SpecBox queries the SDG for likely successor sandboxes, filters out low-value or low-confidence candidates using Lj≥λ and Pi,j≥τ, and then selects a budgeted prefetch set 𝒜i before step N+1 is fully committed.

Refer to caption

Figure 7.Reuse-aware data transmission in SpecBox. A semantic-cache hit returns a prior result (paper_search); a miss executes the sandbox (paper_slides) and delivers the result through the out-of-band data path.

3.3 Reuse-Aware Data Transmission

Once an invocation is ready, its remaining critical-path cost lies in artifact movement (Tdata_io(N)) and tool execution (Tsandbox_exec(N)). SpecBox first checks if it can reuse a cached result; otherwise, it runs the tool and returns its artifact via a separate data path. Fig. 7 details this procedure.

Semantic Caching. Multi-turn agents often re-access unchanged documents, submit semantically equivalent queries with different wording, or re-run deterministic computations. Exact argument matching is robust but fails to capture such superficial variations, while unconstrained semantic matching can mistakenly merge requests that differ in tools, inputs, or side effects. Thus, caching must broaden the set of reusable requests while strictly preserving functional equivalence and behavioral compatibility.

For each completed deterministic invocation, SpecBox stores a normalized invocation signature and its result. During lookup, it first filters cache entries by tool identity, then compares normalized invocation representations are compared against cached signatures. For a request x, result reuse is allowed only if a compatible cache entry exists that satisfies the semantic equivalence constraint:

where x denotes the incoming tool invocation request and xi denotes the i-th cached invocation. Function ϕ⁡(⋅) transforms an invocation into a normalized semantic representation by first removing superficial variations in argument representation and then extracting semantic features for similarity comparison. Function sim(⋅,⋅) measures the similarity between two invocation representations, and the threshold τc controls the strictness of semantic reuse. The tool identity constraint guarantees interface-level compatibility, while the similarity threshold limits reuse to invocations with sufficiently similar normalized semantics.

This design treats semantic matching as a conservative extension of exact reuse rather than an unconstrained approximation mechanism. Semantic similarity only enlarges the reusable request space within the boundary of the same tool interface and deterministic invocation behavior. On a cache miss, execution proceeds along the standard path and appends the fully computed result to the cache. On a cache hit, both sandbox initialization and tool invocation are skipped, thereby reducing Tenv_prep(N) and Tsandbox_exec(N). As will be demonstrated in § 5.3.3, the semantic caching strategy can recover a larger proportion of redundant computation than using the strategy of exact matching alone, while maintaining a conservative fallback path that preserves the inference correctness.

Out-of-Band Data Transmission. Even after a sandbox is fully initialized, conventional RPC transports serialize large logs, files, images, and structured outputs into request–response messages. Consequently, Tdata_io(N) scales with the payload size and blocks the agent prior to its subsequent reasoning step. A wholesale replacement of the control protocol would compromise compatibility with existing MCP tools; therefore, SpecBox instead decouples control signaling from artifact transfer.

The in-band control plane continues to carry standard tool directives, completion notifications, error reports, and compact metadata. Large artifacts are represented on this plane solely by a fixed-size reference and are exchanged via a co-located, zero-copy data path. Under this design, a cache miss executes as usual, publishes its artifact exactly once, and returns a reference to the Agent Engine; a cache hit simply reuses and returns the previously published artifact over the same data path. This architectural decoupling maintains the existing control interface while eliminating RPC-layer serialization and memory copying for high-volume outputs. As demonstrated in § 5.3.4, the corresponding transmission latency becomes effectively insensitive to payload size.

4 Implementation

Refer to caption Figure 8.Overview of SpecBox.

SpecBox is implemented as a predictive serving runtime between the agent engine and external execution sandboxes. Figure 8 shows its control and data paths. The implementation observes an agent run-loop without changing its tool-facing interface, and overlaps runtime work with the LLM and sandbox work already in progress.

Control Plane. On the control plane, a controller subscribes to the incoming token stream and records completed tool transitions. The Keyword Router matches the stream against tool keyword profiles and emits a candidate when the number of matched tool-specific keywords reaches γ=2, the operating point selected in Section 5.3. In parallel, the Semantic Router uses the sparse retrieval configuration selected in that section. Union Assembly dispatches their candidate sets independently to the sandbox manager, which deduplicates requests and starts the corresponding containers. A background prefetch worker updates the sandbox dependency graph from execution traces, applies the transition probabilities in Section 3.2, and submits budgeted next-step warmups while the current tool is executing. In our implementation, we configure the cost threshold as λ=5. prefetch probability threshold as τ=0.6 and the per-step prefetch budget as B=1.

Data Path. Before scheduling a deterministic invocation, the runtime constructs its invocation representation, restricts lookup to the same tool identity, and checks its semantic-result cache. Each cache entry contains the tool identifier, normalized invocation signature, semantic embedding, and result reference. The cache reuses a previous result only when the tool identity matches and the semantic similarity exceeds the threshold τc=0.8 used in the reported experiments. On a miss, the sandbox writes a large result into a host-managed memory-mapped shared-memory region. The control plane carries only a 64-bit token_id that names this region, while the Agent Engine reads the result directly from the shared-memory backplane. Small directives, metadata, completion notifications, and errors remain on the normal control plane. This implements the common cache-or-execute flow in Figure 7 without serializing bulky results through the RPC stack.

Execution boundary and correctness. We consider multi-tenant agents whose tools run in OS-isolated containers or microVMs. Predictive preparation makes an environment ready but does not execute side-effecting work before the agent commits the invocation; unused warmups are discarded. Cache reuse is limited to deterministic, compatible tool requests. The shared-memory bridge operates inside one trusted host or securely managed cluster boundary, with existing access controls preventing cross-tenant memory access. Sandbox escapes, malicious agents, and compromised infrastructure are outside this work’s threat model.

5 Evaluation

5.1 Experiment Setup

Hardware and Software Environment. All experiments are conducted on a commodity server equipped with an 16-core CPU, 256 GiB of host memory, and a 2 TB NVMe SSD. SpecBox is built upon the AgentScope (Gao et al., 2024) framework, specifically integrated with its agent engine layer. The isolated multi-tenant execution sandboxes are instantiated via Docker containers, and the entire runtime infrastructure is implemented in Python. To drive agent reasoning, we utilize Alibaba DashScope’s Qwen3.5-Max model (Alibaba Cloud, 2026), accessed concurrently via its production cloud API endpoints.

Workloads. We evaluate SpecBox using a trace-level benchmark with 200 multi-turn trajectories, derived from MCPBench (Wang et al., 2025) with 32 open-source MCP-compatible tool servers (e.g., Playwright (Microsoft, 2026b), Jupyter (Datalayer, 2026), Neo4j (Neo4j, 2026)) collected from GitHub (GitHub, 2026). To ensure tool-level validity and realistic workflow execution dependencies, we construct the benchmark in two stages:

  • Atomic Tool-use Generation: We generate 20 single-turn interaction templates per tool (640 in total) to cover diverse atomic behaviors across all 32 tools.

  • Multi-turn Trajectory Construction: An LLM planner incrementally generates 1–10 step agent sessions, where each planned task is executed against actual servers to get real execution results back into subsequent planning steps.

The resulting dataset is uniformly distributed with 20 traces per trajectory length, averaging 6.4 steps per session and 2.96 tools per step. This execution-grounded approach ensures tool-level validity and eliminates tool-specific bias while reflecting representative LLM agent workflows.

Methodology. Unless otherwise specified, we evaluate all systems by replaying full traces as session-level workloads with a fixed random seed (seed = 0). Each trace is executed in a step-wise manner, preserving dependencies across reasoning, tool invocation, and sandbox execution to faithfully model realistic multi-turn agent workflows. Under this deterministic sampling setting, the resulting trace distribution naturally exhibits a concentration in the 5–8 step range, which we therefore treat as the representative workload regime rather than a manually selected subset.

We evaluate all experiments at the session level unless explicitly stated otherwise, and defer workload-specific configurations (e.g., trace length or sampling range) to each individual ablation study.

Figure 9.Cumulative sandbox provisioning latency across multi-turn agent sessions (5–8 steps per session).

Figure 10. End-to-end performance and resource consumption under concurrent workloads: (a) P99 E2E latency, (b) mean cumulative sandbox provisioning latency, (c) peak CPU usage, and (d) peak memory usage.

Metrics. We evaluate SpecBox across three dimensions aligned with system design goals:

  • End-to-end Latency: Per agent session, we report mean and tail latency (P99) to capture both average performance and long-tail behavior under multi-step execution.

  • Resource Efficiency: We measure peak CPU and memory consumption under multi-tenant workloads to quantify the runtime overhead introduced by sandbox provisioning, execution, and lifecycle management.

  • Prediction Accuracy: We evaluate the accuracy of all predictive runtime optimizations, including intent-aware sandbox prewarming, stochastic sandbox prefetching, and semantic cache. For each, we report i) correct triggering or retrieval decisions under uncertain tool intent, and ii) false positive rate, reflecting unnecessary or incorrect activations such as misrouted intents, redundant prewarming.

5.2 End-to-End Performance

This section evaluates the end-to-end (E2E) performance of SpecBox under two representative baselines in agent serving. We define our baselines as follows:

  • Reserved Runtime: Maintains permanently warm sandboxes for all candidate tools. This represents the latency lower bound (performance ceiling). However, it is financially and physically non-viable in production due to prohibitive idle memory footprint across massive, multi-tenant MCP tool ecosystems.

  • On-demand Runtime: Instantiates sandboxes dynamically upon tool calls, mirroring production MCP runtimes (Amazon Web Services, 2026; Microsoft, 2026a; Google Cloud, 2026). Since its bottleneck stems from application-level tool binding and session handshakes rather than generic OS booting, it remains the standard practical baseline.

We aim to answer a fundamental question: Can SpecBox successfully decouple execution latency from physical resource constraints, achieving serverful-like performance with serverless-like cost?

5.2.1 Cumulative Sandbox Provisioning Latency in Multi-Turn Agent Workflows

We evaluate the end-to-end impact of SpecBox on sandbox provisioning latency over multi-turn agent execution traces. We define cumulative sandbox provisioning latency as the sum of sandbox initialization delays incurred at each tool invocation within a session, focusing exclusively on environment setup overheads and excluding in-sandbox computation time.

Fig. 9 shows the distribution of cumulative latency across sessions. The On-demand baseline exhibits steadily increasing latency over longer execution horizons due to repeated cold-start overheads, resulting in a heavy-tailed distribution. In contrast, SpecBox significantly reduces cumulative latency and maintains a much tighter distribution concentrated in the sub-second range. Compared to the On-demand baseline, SpecBox achieves a 4.53× reduction in cumulative sandbox provisioning latency. Against the Reserved baseline, SpecBox remains within a 10.6% performance gap while avoiding the substantial resource overhead of persistent sandbox allocation. These results demonstrate that predictive orchestration can effectively approximate Reserved execution performance under a On-demand deployment model.

5.2.2 Scalability

Fig. 10(a) summarizes the scalability of the three runtimes under different concurrency, with QPS increased from 1 to 20. Across low-to-mid concurrency levels, SpecBox consistently remains a latency profile close to Reserved while reducing end-to-end delay compared with On-demand. At higher concurrency, the advantage over On-demand remains substantial: at QPS=20, Laplace achieves 88.7s P99 E2E latency, a 2.9× speedup over On-demand (257.2s). This indicates that SpecBox can absorb increasing concurrency without inheriting the long-tail latency explosion of the On-demand baseline.

Fig. 10(b) also reveals that the main scalability bottleneck of On-demand lies in cumulative sandbox provisioning. In the low-QPS regime, the mean cumulative sandbox provisioning latency stays below 4 seconds for all three modes. However, once the load reaches the higher-QPS regime (QPS≥5), On-demand suffers growing degradation from network contention and resource constraints, and its cumulative sandbox provisioning latency rises steadily and eventually surpasses 50 seconds. By contrast, Laplace remains tightly bounded in the sub-5-second range and continues to slightly outperform Reserved in provisioning latency, reflecting the benefit of SpecBox’s intent-aware sandbox prewarming and stochastic sandbox prefetching under concurrency pressure.

5.2.3 Resource Footprint and Efficiency

Fig. 10(c) shows that the CPU footprint of the three runtimes remains relatively compact under low-load conditions but diverges significantly as concurrency scales. While SpecBox exhibits no obvious advantages under low QPS loads, its strengths gradually emerge in the high-QPS regime where QPS≥8. It consistently restrains peak CPU utilization across all tested loads, with the peak resource consumption capped at around 12.2 cores. This translates to a 22.8%–23.3% reduction in peak CPU pressure compared to both On-demand and Reserved, which demand roughly 15.8–15.9 cores. Particularly in the high-QPS regime, SpecBox stabilizes within a tight band of ∼11–12 cores, whereas the two baselines fluctuate at a higher plateau of 14–16 cores, demonstrating that SpecBox effectively mitigates CPU contention (saving up to 25% CPU resource under peak load) without sacrificing sub-millisecond control-plane responsiveness.

The memory footprint (Fig. 10(d)) exhibits a pronounced resource separation across the evaluation spectrum. As expected, Reserved is the most memory-intensive variant due to its persistence strategy; its memory consumption surges from an initial 24.6 GiB at light load to a massive peak of 80.6 GiB, staying sustained above 60 GiB as concurrency intensifies. Conversely, On-demand maintains a lean profile, bounding its peak usage between 24.3 GiB and 40.4 GiB. Notably, SpecBox closely mirrors the trajectory of On-demand across the majority of QPS configurations, topping out at 49.4 GiB. This represents a 45.9% reduction in peak memory footprint compared to Reserved, demonstrating that SpecBox successfully eliminates the prohibitive memory holding costs of long-run sandboxes, while maintaining a predictable, serverless-like resource elasticity under heavy concurrent workflows.

5.3 Micro-Benchmarking

This section provides mechanism-level attribution for the end-to-end improvements reported above. In particular, we isolate the effects of intent-aware sandbox prewarming, stochastic sandbox prefetching, semantic caching, and out-of-band data transmission to avoid conflating component contributions in the E2E section.

5.3.1 Effectiveness of Intent-Aware Sandbox Prewarming

We further decompose intent-aware sandbox prewarming into three orthogonal factors: the keyword trigger threshold, the semantic model family, and the hybrid fusion policy. All results are measured on the same sampled tasks and repeated 100 times.

Sensitivity of Keyword Router Threshold γ.

To rigorously isolate the impact of the keyword matching threshold, we evaluate γ∈{1,2,3} under identical workload profiles, executing 100 independent trials for each configuration to ensure statistical convergence. As quantified in Table 1, γ=2 emerges as the optimal configuration for the Keyword Router. Specifically, γ=2 substantially compresses the average keyword-driven waiting latency to 323.0 ms (a 2.43× reduction compared to 786.6 ms under γ=1) while maintaining a high predictive routing coverage of 95%.

The severe latency degradation under γ=1 is primarily driven by its hyper-sensitivity, which triggers 20 distinct tool mismatch instances across the 100 runs; these false positives inadvertently amplify cold-start fallback penalties and introduce heavy tail latencies. Conversely, while increasing the threshold to γ=3 further sharpens routing precision (yielding a 97% match rate with only 3 mismatch instances), it severely erodes the prewarm lead time, causing the average waiting time to climb back to 621.8 ms. This empirical trade-off confirms that γ=2 effectively balances early predictive agility with resource stability.

Table 1.Keyword threshold sensitivity (γ=1,2,3).

γ Avg Wait (ms) Match Rate Mismatch Runs
1 786.619 80.0% 20
2 323.021 95.0% 5
3 621.812 97.0% 3
Sensitivity of Semantic Router Models

Table 2 compares the performance and computational trade-offs of three semantic routing models under their optimal configurations:

  • Retrieval: This model is implemented as a non-neural, sparse token-level retriever. It leverages TF-IDF (Sparck Jones, 1972) weighting combined with a Top-N nearest neighbor aggregation network, constructing composite vector features across token unigrams, selective bigrams, and character n-grams (ranging from 3 to 5 characters).

  • Encoder: This neural model is built upon a fine-tuned all-MiniLM-L6-v2 (sentence-transformers, 2026) transformer sequence representation network. It maps fluid multi-turn trajectories into dense vector spaces via contrastive pair-wise optimization under a CosineSimilarityLoss constraint.

  • FastText: This lightweight model (Joulin et al., 2017) represents text as the average of word and subword embeddings followed by a linear classifier.

Empirically, as quantified in Table 2, both Retrieval and Encoder models achieve a optimal prewarming Hit Rate, demonstrating that both sparse lexical tokens and dense semantic features can successfully capture all necessary tool invocation targets. However, Retrieval achieves higher orchestration quality and routing precision, yielding a Micro-F1 of 0.970 and a Precision of 0.942, whereas Encoder degrades to a Micro-F1 of 0.776 and a Precision of 0.634.

Furthermore, from an runtime efficiency perspective, Retrieval operates with a significantly lower computational footprint, executing with an average inference latency of only 2.116 ms–yielding a 3.70× speedup compared to the 7.822 ms latency incurred by Encoder. Conversely, while FastText provides ultra-low execution latency (0.171 ms), its shallow token representation space fails to deliver usable discrimination under multi-turn reasoning context drifts, resulting in a Micro-F1 and Hit Rate of merely 0.124. Consequently, Retrieval is selected as the production instance for Laplace’s semantic branch, as it strikes the optimal Pareto-efficiency frontier between high-fidelity prediction accuracy and low-overhead control-plane latency.

Table 2.Semantic router model comparison (best operating point per model).

Model Micro-F1 Hit Rate (Top-3) Precision Avg Latency (ms)
Retrieval 0.970 0.992 0.942 2.116
Encoder 0.776 0.968 0.634 7.822
FastText 0.124 0.124 0.124 0.171
Sensitivity of Assembly Policies.

To rigorously isolate the algorithmic impact of the intent combination layer, we evaluate three distinct fusion policy paradigms under identical multi-turn context distributions:

  • Union (∪): The predictive prewarming primitive is non-blocking and asynchronously dispatched the exact microsecond either the keyword router or the semantic router hits their respective individual activation boundaries.

  • Intersection (∩): Predictive orchestration strictly enforces a dual-router consensus; a sandbox is prewarmed if and only if both the keyword router and semantic router concurrently validate the sandbox candidate’s invocation intent.

  • Weighted: This hybrid policy aggregates multi-modal intent metrics into a centralized candidate score: s⁡(c)=wk⋅sk(c)+ws⋅ss(c), where sk(c) denotes the matched-keyword ratio (normalized against the static lexicon scale) and ss(c) represents the real-time semantic retrieval similarity vector. A predictive trigger is dispatched only when clearing a rigid threshold: s⁡(c)≥τf. In our empirical runner, we implement a symmetric baseline configuration with wk=0.5,ws=0.5, and τf=0.5.

As quantified in Table 3, there is a clear trade-off between the aggressive Union Assembly and the conservative Intersection policy. Union Assembly prioritizes latency-masking agility, reducing the average waiting latency to just 124.45 ms at the cost of a slight 5.0% cold-start ratio. In contrast, the conservative Intersection policy eliminates cold starts (0.0%) by enforcing strict dual-router consensus, but at the cost of blocking the critical path and driving the average latency up to 393.52 ms (3.16× higher than Union).

The Weighted policy performs the worst, inflating latency to 1308.38 ms due to a 45.0% cold-start ratio. This failure stems from the structural misalignment between keyword tokens and high-dimensional semantics. Without dynamic re-normalization, shifting context distributions under multi-turn reasoning cause the combined scores to drift, frequently failing to clear the static τf=0.5 threshold. These results validate our choice of a Union-first assembly design to maximize latency-masking performance while maintaining practical routing correctness.

Table 3.Assembly policy trade-off.

Assembly Mode Avg Wait (ms) Target Match Rate Cold Start Ratio
Union 124.446 95.0% 5.0%
Intersection 393.523 100.0% 0.0%
Weighted 1308.378 55.0% 45.0%

5.3.2 Effectiveness of Stochastic Sandbox Prefetching

Figure 11.Dynamic execution profiling across a 10-turn conversation horizon, benchmarking per-turn average waiting latency (left) against the corresponding average cold-start sandbox activation count (right).

To isolate the contribution of stochastic prefetching under multi-turn agent execution, we evaluate Laplace with two deployment variants under an identical planning workload:

  • SpecBox-Reactive: A routing-only baseline that performs inline token-level intent detection but does not use cross-step transition prediction.

  • SpecBox-Proactive: The full design that couples online hybrid routing with a stochastic Markovian prefetcher over the sandbox dependency graph (SDG), enabling asynchronous sandbox preparation before the next step.

Fig. 11 reports per-turn average waiting latency and cold-start counts over a 10-turn horizon. During Turn 1, both variants show similar latency (512.06 ms vs. 540.06 ms), since no historical transition signal is available to initialize the Markov predictor. From Turn 2 onward, the two trajectories diverge sharply. SpecBox*-Proactive* reduces average waiting latency from 540.06 ms to 138.14 ms at Turn 2 and further to 97.14 ms by Turn 10, while SpecBox*-Reactive* increases to 583.26 ms at Turn 10. This corresponds to a 6.0× end-horizon latency reduction. The mechanism is consistent with the cold-start telemetry. Under SpecBox*-Reactive*, average cold-start count peaks at 2.90 instances per turn (Turn 9), indicating repeated exposure to sandbox initialization cost on the critical path. In contrast, SpecBox*-Proactive* keeps per-turn cold starts within 0.24–0.83, effectively masking startup latency through early scheduling. These results show that Laplace improves multi-turn responsiveness through accurate, low-overhead temporal prefetching rather than infrastructure over-provisioning.

5.3.3 Effectiveness of Semantic Cache

To quantify the impact of our semantic cache, we benchmark three variants under the same repeated-request workload over 100 runs:

  • No Cache: that executes every repeated request from scratch.

  • Exact-Match Cache: A strict cache that matches on tool identity and normalized argument equality.

  • Semantic Cache: reusing results when the tool identity matches and semantic similarity exceeds the threshold.

Table 4.Semantic cache trade-off.

Setting Avg Wait (ms) Hit Rate Bypass Ratio
No Cache 412.78 0.0% 0.0%
Exact-Match Cache 233.64 33.6% 100.0%
Semantic Cache (τ=0.6) 141.93 37.4% 84.8%

As shown in Table 4, while exact-match caching improves performance over No Cache, semantic caching captures a wider envelope of near-duplicate requests by tolerating surface-form variations. Compared to No Cache, semantic caching achieves a 2.91× speedup in average waiting latency (dropping from 412.78 ms to 141.93 ms) and increases the cache hit rate to 37.4% .

However, sematic caching is slightly lower than exact matching because a small fraction of semantic hits are not sufficiently reliable to skip sandbox setup and must fall back to normal execution after validation. This behavior is consistent with the intended design: semantic equivalence expands the reusable request space, but some loose matches still require conservative verification before execution can be bypassed.

5.3.4 Effectiveness of Out-of-Band Data Transmission

Figure 12.Data transmission latency scaling profiles under varying payload sizes, comparing SpecBox’s out-of-band data transmission against standard JSON-RPC serialization.

To evaluate the architectural efficiency of our control-and-data plane separation, we isolate the data transmission overhead by benchmarking two distinct transport paradigms across an exponential data payload spectrum scaling from 1.00 MB to 1000.00 MB:

  • Out-of-Band: Our proposed out-of-band transport mechanism. It completely bypasses the control-plane RPC tunnel by writing bulky state payloads directly to a dedicated local shared-memory substrate or zero-copy virtualized host-guest ring buffers, passing only lightweight, fixed-size references over the wire.

  • JSON-RPC: The standard baseline paradigm utilized in conventional agent runtimes. It marshals multi-modal data payloads directly into the inline runtime execution stream, forcing the control-plane to serialize and transport raw data matrices synchronously via text-based JSON-RPC network primitives.

As shown in Fig. 12, the empirical measurements reveal a clear scaling divergence between the two protocols across an exponential payload spectrum. At the baseline threshold (1.00 MB), both configurations demonstrate sub-3 millisecond performance, with Out-of-Band maintaining a slight edge (1.95 ms vs. 2.28 ms). However, as the payload expands, the JSON-RPC pipeline suffers a catastrophic linear performance degeneration; throttled by heavy serialization bottlenecks, its latency grows to 132.45 ms at 100.00 MB and reaches 1873.16 ms at the 1000.00 MB boundary. Conversely, Out-of-Band demonstrates a near-constant O⁡(1) scaling profile, drifting to only 5.97 ms at the 1 GB boundary—a 313.55× latency reduction.

This performance gap stems from eliminating critical-path serialization and memory copying. In multi-turn workflows, agents frequently exchange large multi-modal states like high-dimensional vectors or media files. Standard JSON-RPC stalls the control loop due to synchronous serialization, whereas SpecBox’s out-of-band design decouples control signals from raw payload routing, reducing data transfer to a constant-time reference-passing operation. These results demonstrate that separating the control and data planes keeps SpecBox’s orchestration overhead minimal and independent of payload size.

6 Discussion

Architectural Overhead. A critical concern in predictive serving runtimes is whether the system-level orchestration mechanisms introduce non-negligible processing penalties onto the critical path. In SpecBox, this control-plane overhead is thoroughly isolated from the GPU-bound inference loop through strict out-of-band execution and parallel routing mechanics. While the token-level scanning runs with deterministic 𝒪⁡(1) complexity, the inherently heavier semantic retrieval engine is offloaded to dedicated background CPU worker threads. Because the intent router utilizes an Union Assembly (∪), the critical path never blocks for late-arriving semantic embeddings; any early fast-path trigger instantly dispatches the activation primitive within microseconds. Combined with the prefetching daemon–which evaluates low-dimension Markov matrix transitions bounded by the small cardinality of active sandboxes–SpecBox restricts its control-plane telemetry to tens of microseconds, securing a nearly zero-cost latency impact on token generation.

Ecosystem Generalization. We position SpecBox as a runtime middleware layer operating between upstream reasoning agents and downstream execution environments. This design leverages two widely available capabilities in modern agent ecosystems (Wu et al., 2023; Langchain-ai, 2026; Gao et al., 2024): streaming token-level outputs from upstream frameworks, and standardized tool interfaces provided by protocols such as MCP (Anthropic PBC, 2024). As a result, SpecBox requires no additional system-level constraints beyond what is already supported in existing agent and sandbox orchestration stacks.

Importantly, this execution abstraction extends beyond inference-time serving to agentic reinforcement learning (Agentic RL) frameworks, such as VERL (Sheng et al., 2024) and Slime (Zhu et al., 2025), where rollout generation and environment interaction follow a structurally similar loop. In these Agentic RL settings, SpecBox ’s runtime optimizations—namely intent-aware sandbox prewarming and stochastic sandbox prefetching—can be seamlessly adapted to mask environment initialization and interaction overheads during training rollouts with modest integration effort.

Limitations and Future Work. Despite its performance gains, SpecBox exhibits certain architectural boundaries. On the control plane, our prefetcher assumes first-order history dependence via a Markov Chain model, which may suffer from diminished prediction accuracy during open-ended, long-horizon agent workflows. Crucially, SpecBox inherently mitigates this via its hierarchical, two-tiered preparation: any inter-step prefetch miss gracefully falls back to the intra-step intent-aware prewarming during real-time token streaming. This design guarantees that worst-case environment latency remains tightly bounded within the LLM’s decoding phase. Future work will investigate online Graph Neural Networks (GNNs) to adaptively capture non-linear transition patterns.

On the data plane, our mmap-based transmission requires co-locating the Agent Engine and sandboxes within the same host boundary. While this design seamlessly aligns with mainstream multi-tenant serving topologies (e.g., Kubernetes Pod IPC sharing or Sidecar patterns) and leverages robust container-level namespace and cgroup security isolation, it restricts single-session cross-node scaling. To support large-scale distributed clusters, we plan to integrate RDMA-assisted zero-copy transmission, extending our out-of-band data plane into disaggregated cloud infrastructures.

Lightweight Sandbox Runtimes. The systems community has actively explored lightweight isolation mechanisms to mitigate the physical instantiation cost of serverless execution environments. Notable advancements include micro-virtual machines (MicroVMs) like Firecracker (Agache et al., 2020) and RunD (Li et al., 2022), WebAssembly (Wasm) (WebAssembly, 2026) runtimes, and process-level snapshotting/forking frameworks such as FaaSnap (Ao et al., 2022) and TrEnv-X (Huang et al., 2026). These infrastructure-level systems focus on minimizing localized host setup or exploiting hardware-assisted remote memory pools (e.g., CXL/RDMA) to share and reuse physical sandboxes across tenants.

Crucially, these low-layer container optimizations are entirely orthogonal to SpecBox. While microVM snapshots or repurposable sandboxes successfully compress the physical infrastructure boot time to milliseconds, application-layer handshakes (such as MCP discovery hooks) and environment attachment bottlenecks still natively linger on the critical runtime path. SpecBox operates at a higher, application-perceptive orchestration layer; it complements these physical speedups by exploiting a distinct temporal dimension—overlapping control-plane scheduling with streaming token generation—to fully mask, rather than compress, the intrinsic readiness latency.

Serverless Predictive Prewarming. Predictive prewarming is a widely embraced technique in standard serverless frameworks to eradicate the notorious tail-latency penalties of tenant cold starts. State-of-the-art prefetching daemons (e.g., Mitosis (Wei et al., 2023) and IceBreaker (Roy et al., 2022)) primarily depend on historical time-series analytics, statistical invocation frequency histograms, or temporal correlation clustering to predictively prepare idle container runtimes.

However, these classical prewarming paradigms inherently assume that incoming requests follow independent and identically distributed arrival models or deterministic time-triggered patterns. This core assumption breaks down completely under emerging LLM Agent workloads. In multi-turn autonomous agent reasoning loops, tool invocations are dynamically, autonomously, and non-linearly determined by the LLM’s fluid context trajectory, presenting complex execution dependency horizons. SpecBox bridges this gap by co-designing the serving infrastructure with explicit agent behavioral characteristics, introducing a stochastic Markovian predictive framework built over a sandbox dependency graph (SDG) to model real-time autonomous state transitions.

LLM Serving and Agent Orchestration. Accelerating the end-to-end execution of Large Language Models has driven extensive research across the AI systems spectrum. On the model-serving boundary, mainstream runtimes such as vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024) optimize GPU-internal kernel execution, memory caching via PagedAttention, and automated speculative decoding. Parallel to this, application-level agent orchestration platforms like AgentScope (Gao et al., 2024), AutoGen (Wu et al., 2023) and LangGraph (Langchain-ai, 2026) provide modular abstractions for programming multi-turn autonomous multi-agent teams.

Nevertheless, existing LLM serving engines fundamentally treat model execution as a self-contained GPU computing unit, entirely oblivious to the physical host-plane environment friction when interacting with external tools. Conversely, high-level agent frameworks lack system-level visibility into cloud-native host topologies, resulting in uncoordinated data transfer and execution stalls. SpecBox operates as a runtime middleware layer that bridges this gap, enabling co-optimized out-of-band control-plane predictive routing and shared-memory data transmission across heterogeneous multi-tenant host execution boundaries.

8 Conclusions

We present SpecBox, a predictive execution runtime for LLM-based agent systems that reduces tail latency and resource inefficiency in multi-tenant environments. By analyzing the end-to-end agent execution step, we find that inefficiencies primarily arise from rigid dependencies among reasoning, environment initialization, and sandbox execution. SpecBox addresses this through a unified design that enables temporal overlap between LLM execution and sandbox setup, anticipates future sandbox needs across steps to reduce cross-step cold-start overheads, and eliminates redundant computation and communication through reuse-aware execution and out-of-band data transport. SpecBox is implemented on top of AgentScope (Gao et al., 2024) and evaluated it under highly concurrent multi-turn workloads. Results show up to a 2.9× reduction in P99 latency and 45.9% lower peak memory usage, with a 97.9% prewarming hit rate. These results demonstrate that exploiting temporal overlap across the agent execution loop is an effective and general mechanism for improving efficiency in large-scale agent deployments.

3.3 - 论文中文翻译

论文的中文翻译版本

SpecBox：面向高效 LLM 智能体服务的推测性沙盒调度

摘要

随着 LLM 智能体日益依赖模型上下文协议（Model Context Protocol, MCP）来调用隔离的外部沙盒，解耦式沙盒部署在资源利用率与交互式尾延迟之间引入了根本张力。持久的长生命周期沙盒预留会在规模化时带来过高的内存开销，而惰性按需实例化又会产生严重的冷启动惩罚，从而在多租户、多轮智能体工作负载下劣化响应性能。为化解这一两难，我们提出 SpecBox——一个围绕推测性沙盒预分配构建的运行时，专为动态 LLM 智能体执行流水线设计。

其核心是：SpecBox 实现关键词匹配与流式语义嵌入，以支持意图驱动的沙盒预热；该机制在 LLM Token 生成过程中识别即将到来的工具执行需求，并将沙盒启动与模型推理完全重叠。为将预热窗口延伸到后续智能体步骤，框架在沙盒依赖图之上利用上下文感知的随机预取，在执行之前以概率方式预测未来的沙盒切换。我们还辅以两项正交优化：语义结果缓存，用于裁剪冗余的重复沙盒调用；以及专用的带外共享内存传输平面，绕过传统网络序列化，实现产物的零拷贝传输。在高并发多轮智能体轨迹上的评估表明，相对按需沙盒基线，SpecBox 可将 P99 端到端延迟最多降低 2.9×，同时相对永久预留沙盒部署，峰值内存消耗下降 45.9%。

关键词：LLM 智能体、执行运行时、预测性预热

1 引言

现代 LLM 智能体正从传统的文本生成器演进为可迭代推理、规划并执行外部动作的自主系统（Yao et al., 2022；Li et al., 2024b；Wang et al., 2024）。与主要优化 Token 生成的传统 LLM 服务工作负载不同（Agrawal et al., 2024；Zhang et al., 2025），智能体执行形成有状态的运行循环：基于 LLM 的控制器反复调用外部工具（Microsoft, 2026b；GitHub, 2026；Datalayer, 2026；Neo4j, 2026），例如代码执行、网页自动化与数据处理服务。出于安全、隔离与可复现性，这些工具正越来越多地部署为独立沙盒环境，并通过模型上下文协议（MCP）（Anthropic PBC, 2024）等标准化接口访问。因此，智能体执行的延迟不再仅由模型推理决定，还取决于协调异构外部执行环境的效率。

现代云原生基础设施日益采用无服务器沙盒执行，以支撑大规模并发智能体会话（Amazon Web Services, 2026；Google Cloud, 2026；Microsoft, 2026a），即执行环境按需初始化，而非作为长生命周期实例预留。这一设计为多租户智能体工作负载提供了所需弹性，但引入了不可忽视的延迟（Puliafito et al., 2022；Yu et al., 2024；Xu et al., 2024；Stojkovic et al., 2023a）。每次工具调用都可能产生沙盒初始化开销——如镜像加载、文件系统准备、命名空间配置与运行时握手——从而在执行开始前引入秒级延迟（Huang et al., 2026）。在多轮智能体工作流中，这类启动开销会在连续工具调用间累积，显著劣化端到端响应性，并限制交互式智能体服务的实用性。

图 1. 主动重叠执行与原生朴素方法的对比。

这一延迟的主要原因并非沙盒初始化本身，而是事实标准智能体运行时（如 AgentScope（Gao et al., 2024）、AutoGen（Wu et al., 2023）与 LangGraph（Langchain-ai, 2026））的朴素实现所采用的串行执行模型。这些系统使用反应式、无流水线编排的串行模型：沙盒准备阶段仅在 LLM 推理完成 Token 生成、工具调用被完全确定之后才开始。因此，环境准备完全暴露在关键路径上，在模型推理与工具执行之间留下空闲时段。当 GPU 忙于生成 Token 时，CPU 侧资源利用不足——一旦沙盒初始化开始，GPU 又必须等待外部执行完成。如图 1 所示，反应式模型将 LLM 推理与环境准备串行化，未能有效地将计算与环境供给重叠。

现有运行时优化，如推测执行（Stojkovic et al., 2023b；Li et al., 2024a）、预热（Mahgoub et al., 2022；Sui et al., 2024）、工作流编排（Liu et al., 2023；Li et al., 2023）与缓存（Xu et al., 2024；Lu et al., 2024），在无服务器与云系统中已有充分研究。然而，它们并不能直接迁移到 LLM 智能体运行时，因为智能体执行与传统请求处理存在根本差异。既有工作通常假设固定的执行目标，或在运行前已有预定义工作流图。相反，自主智能体工作流是通过自回归推理在线产生的：工具调用从流式 Token 生成中逐步涌现，直到足够的语义上下文可用之前都保持不确定。多轮会话还会依据中间观测与结果进一步演化，使未来的工具选择既依赖工作流又高度动态。这些特征使得对现有技术的简单扩展并不可行。

因此，在 LLM 智能体运行时的语境下，关键思路是在沙盒被请求之前预先启动最可能需要的沙盒，使环境准备与正在进行的 LLM 生成重叠。然而，将这一思路落实到自主智能体系统会带来三项独特挑战。第一，在单步（智能体迭代）内部，意图必须从仅具部分语义的早期 Token 流中推断；使用短上下文、低阈值或宽候选集会改善重叠，但会带来假阳性预热以及内存/CPU 浪费的风险。第二，跨步骤预测未来将调用哪个工具，会随预测窗口拉长而越来越不可靠；若对所有可能后继都预热，成本将退化回预留部署。第三，仅预热只能消除沙盒启动延迟：即便沙盒已就绪，重复的工具执行与大型产物传输仍可能停留在关键路径上。因此，系统必须复用语义等价的执行结果，并将大宗数据传输与控制信令解耦，同时保持协议兼容性与用户可见语义。

本文提出 SpecBox，一个对智能体工作流任务进行高效编排的预测性服务系统。SpecBox 将 LLM 执行与环境准备重叠，打破朴素智能体运行时实现中僵硬的串行依赖。SpecBox 通过三项协同技术降低端到端延迟：i）意图感知沙盒预热，基于流式 Token 输出推测执行意图，在恰当时机启动，并在同一步内将沙盒准备与正在进行的生成重叠；ii）随机沙盒预取，利用历史智能体执行轨迹预判未来沙盒需求，并在当前步骤期间为下一步准备沙盒，从而降低冷启动延迟；iii）复用感知数据传输，利用语义相似性，通过语义缓存绕过冗余工具执行，并通过带外数据路径将大型产物交付与控制信令解耦，消除不必要的计算与序列化开销。SpecBox 的原型已实现并集成到开源 AgentScope 框架（Gao et al., 2024）。SpecBox 与框架无关，可高度扩展到任意 LLM 智能体服务基础设施。在多样化的多轮、高并发智能体基准上，SpecBox 优于现有最先进的无服务器运行时，同时保持工作流正确性与协议兼容性。SpecBox 将 P99 延迟最多降低 2.9×，峰值主机内存使用降低 45.9%。本文的主要贡献如下。

  • 刻画多轮 LLM 智能体执行的延迟关键路径，并提出一种新的沙盒预热方法（C1）：通过推断流式 Token 级意图，使沙盒准备能更好地与正在进行的 LLM 推理重叠（§ 3.1）。

  • 设计随机沙盒预取机制（C2），通过挖掘历史智能体执行轨迹，并在沙盒依赖图上预热可能的沙盒环境，降低跨步冷启动延迟（§ 3.2）。

  • 引入复用感知数据传输（C3），通过带外传输与语义缓存，高效传输并复用中间执行产物，同时消除冗余沙盒初始化（§ 3.3）。

2 背景与动机

2.1 智能体工作流与环境

大语言模型（LLM）应用已从单体、单轮聊天机器人转向分布式自主智能体。传统工作流依赖人工预定义的执行图，操作序列在运行前即已固定。相反，现代智能体工作流融入自主决策：给定高层目标，智能体可动态分解任务、选择执行沙盒或外部环境，并根据中间观测调整轨迹。因此，工作流不再静态指定，而是从 LLM 推理与外部环境的持续交互中涌现（Yao et al., 2022；Li et al., 2024b；Wang et al., 2024）。

智能体工作流通过一系列事件驱动的执行步骤实现，每一步代表涉及推理、规划或调用本地与远程沙盒的中间任务。需要文件系统交互、代码执行、网页访问或对专有数据库查询的任务，会被委托给隔离的外部执行层（环境或工具沙盒）。标准化协议——尤其是 Anthropic 提出的模型上下文协议（MCP）（Anthropic PBC, 2024）——形式化了这一接口边界。MCP 在 HTTP 与 Server-Sent Events（SSE）等传输之上规定了公共通信层，从而将工具调用转化为一组松耦合微服务。这一架构与新兴的面向智能体的操作系统高度契合：后者将 LLM 概念化为中央处理器，将外部环境概念化为外围设备（Mei et al.,；Gao et al., 2024）。从系统工程角度看，智能体执行日益表现为 LLM 推理核心与多个环境服务之间连续的、异构的、类 RPC 交互流，而非单体的、局部化的计算工作负载。

2.2 服务运行时

将智能体引擎与其底层执行环境解耦，能够带来显著的可扩展性与架构灵活性。然而，在云原生、多租户集群基础设施中部署这些解耦组件，会在性能与资源管理方面引入重大挑战。服务提供商必须仔细平衡隔离保证、资源效率与端到端延迟，从而暴露出长运行（预留）与无服务器（按需）服务范式之间固有的权衡。

预留式智能体运行时。 一种直接策略（Tan et al., 2025；Huang et al., 2026）是为每个用户租户及其关联沙盒维持预初始化、始终在线的容器。然而，在包含数千种细粒度工具的当代智能体生态系统中，将所有执行环境保持在预热状态会变得资源成本过高：空闲容器会带来可观的主机内存与 CPU 开销（Xu et al., 2024；Stojkovic et al., 2023a），进而导致显著的集群低利用率与资源干扰，使大规模多租户部署在经济上不可持续。

按需智能体运行时。 为提升资源利用率，现代云原生平台普遍采用按需运行时供给（Du et al., 2020；Shahrad et al., 2020），即仅在智能体显式请求时才实例化（“冷启动”）工具沙盒。然而，这一设计选择会引入显著的尾延迟：按需初始化隔离容器会带来一系列串行开销，包括容器镜像下载与解压、网络命名空间设置、虚拟文件系统挂载，以及应用级握手过程。在突发工作负载或复杂多轮工作流下，这些数秒级的冷启动延迟会沿智能体端到端关键路径累积，从而抬高 P99 尾延迟，并劣化实时智能体的交互式服务质量（QoS）。

2.3 智能体运行时中的执行瓶颈

与传统 LLM 服务不同，智能体会话构成由多次迭代推理与沙盒化执行步骤组成的连续、有状态执行循环。主流智能体框架（Gao et al., 2024；Wu et al., 2023；Langchain-ai, 2026）采用反应式执行范式，即仅在 LLM 完成推理步骤后才发起每次工具调用。这一设计在模型推理与环境交互之间诱导出严格串行的依赖链。所得执行工作流如图 1 所示。

如图 2(a) 所示，单次执行步骤 N 的端到端延迟可分解为若干部分：

前两个分量与 LLM 推理相关，包括上下文处理与 Token 生成。其余三个定义了智能体运行时的优化边界：Tenv_prep(N) 是使所选沙盒环境就绪的时间，Tdata_io(N) 是交换调用输入与结果的时间，Tsandbox_exec(N) 是执行工具所花费的时间。

SpecBox 在不改变 LLM 推理栈的前提下针对这些时间段进行优化：将准备与推理重叠、减少数据移动，并消除冗余执行。我们据此得到如下观察，构成本研究的动机。

观察 #1：步内准备可与 Token 生成重叠。 在当今的反应式执行模型中，即便提示词、计划与部分 Token 流往往已经揭示了可能的沙盒，环境初始化也必须等到 LLM 发出显式沙盒调用后才能开始。利用这些部分证据更早启动准备，可将预热前移到对应调用之前。这一间隔使沙盒启动、运行时连接建立及其他准备工作能够与剩余的 Tgeneration(N) 重叠，而非将全部 Tenv_prep(N) 放在关键路径上。如图 2(b) 所示，沙盒初始化需要数秒，重量级沙盒甚至可接近 20 秒，因此这一重叠机制对最小化系统所观测到的有效准备延迟至关重要。

观察 #2：跨步执行可提前预热。 智能体会话中的连续步骤常常表现出强时间局部性。例如，在大语言模型（LLM）服务工作负载中，论文检索步骤之后往往是处理已检索文档的文档阅读步骤，而数据分析步骤之后通常是图表生成步骤。尽管如此，主流运行时系统一般将每次调用视为独立事件，一旦某步启动便丢弃工作流上下文。相反，运行时可利用当前沙盒的执行窗口，在步骤 N+1 开始之前主动实例化并预热最可能的后续执行环境，从而使步内预热超越当前解码阶段、进一步提前。

观察 #3：冗余执行与过量不必要的数据传输。 在多轮智能体工作流中，后续推理步骤常常回访已经产生的信息，例如再次查询同一文档或重跑确定性分析。然而，即便等价结果已经可用，传统运行时仍会重新执行这些操作，反复支付 Tsandbox_exec(N) 的成本。与此同时，沙盒接口往往将大型中间产物（如文件、图像、结构化输出）的同步传输与控制消息耦合，使 Tdata_io(N) 随产物规模增长。直观上，可以通过缓存并复用先前执行结果，并将产物传输与控制信令解耦，从而加速智能体执行，消除冗余工作与数据传输开销。

这些观察揭示了智能体执行循环中三个优化切入点：在同一步内将环境准备与 LLM 操作重叠；在当前步骤中提前下一步的环境准备；以及消除重复执行或数据移动。现有 LLM 服务框架主要聚焦于优化模型执行——如预填充与解码阶段，以及 KV 缓存管理——而无服务器运行时主要降低容器初始化开销，却不考虑用户侧意图。然而，两者都没有直接回答：面向智能体的运行时如何在利用这些优化的同时，保持资源效率并保留工具调用与使用的既定语义。

图 2. 智能体运行时中的执行瓶颈。(a) 执行步骤延迟分解。(b) 32 个沙盒化环境的平均冷启动延迟，包括全部 MCPBench（Wang et al., 2025）沙盒及其他常用环境。大多数沙盒在 2–4 秒内完成初始化，而资源密集型环境可能需要大约 20 秒。

图 3. 意图感知沙盒预热：规划智能体将用户请求展开为 ReAct 步骤，关键词预测与语义预测共同决定是否预热沙盒。

图 4. 路由策略之间的权衡。关键词路由以潜在假阳性换取早期决策；语义路由与交集会推迟精确决策；并集组装在保留精度的同时，允许任一路由器尽早触发预热。

2.4 研究挑战

设计有效的执行运行时以加速 LLM 智能体服务，面临如下挑战。

挑战 #1：早期意图预测 vs. 资源浪费。 从给定的不完整 Token 流中推测工具使用意图，对于保证沙盒环境准备的及时性、同时避免因意图预测不准而启动不需要的工具至关重要。早期预热必须依赖高度模糊的早期 Token（例如在不同 MCP 工具间复用的通用动词，如 read 与 get），这常常导致假阳性沙盒激活，浪费资源，并可能在多租户部署中触发主机级 OOM。

挑战 #2：跨步提前预热 vs. 非确定性工作流执行。 在下一意图显现之前选定后续步骤的沙盒环境是有利的。智能体工作流以概率方式执行，允许多种合理的后继状态。对所有潜在后继都预热，会带来与完全预留部署相当的计算与资源开销；而只预热单一后继，则会大幅降低摊销沙盒初始化成本的机会，仅带来有限的延迟改善。相反，运行时系统应利用历史转移数据对一小集合高度可能的目标环境排序，再用当前步骤可用的信号精炼这一选择。这在资源利用率与延迟降低之间取得平衡，同时保留将沙盒初始化隐藏在正在进行的计算之后的能力。

挑战 #3：复用与数据路径效率 vs. 沙盒兼容性。 我们必须跳过冗余工作，同时不改变智能体从沙盒调用中观察到的内容。精确复用是安全的，但会错过以不同表述表达的语义等价请求；不受约束的语义复用可能产生不兼容的结果。类似地，将大型产物放入普通 RPC 消息可保持兼容性，但数据移动仍与载荷大小成正比；而替换控制接口则会破坏现有工具。因此，运行时必须检测兼容复用、拒绝不安全近似，并将大宗产物与控制信令解耦，同时保留沙盒执行语义。

3 SpecBox 的设计

我们关注反应式智能体执行中的三个关键时间段：环境准备、数据移动与沙盒执行。本节说明 SpecBox 如何通过解耦而又相互关联的优化来加速它们：意图感知沙盒预热，尽早推断步内工具意图，使环境准备与 LLM 服务的解码重叠（§ 3.1）；随机沙盒预取，利用跨步工作流规律性进一步将准备前移（§ 3.2）；以及复用感知数据传输，避免冗余执行并将大型产物移出控制路径（§ 3.3）。

图 5. SpecBox 中的随机沙盒预取。执行历史预测后续 ReAct 步骤可能需要的沙盒。

3.1 意图感知沙盒预热

朴素的反应式运行时在沙盒准备之前等待完整的工具调用，即便提示词、演化中的计划与部分生成已经揭示了用户意图。SpecBox 则通过推断流式 Token 级意图更早地预热沙盒，使沙盒准备更好地与正在进行的 LLM 推理重叠。

SpecBox 充分利用 LLM 解码时段 Tgeneration(N)，获取刚好足够的用户意图以准确启动沙盒，同时不损害预热的及时性。做法是将流式上下文送入两个独立路由器。如图 4 所示，存在一个两难：更早的意图预测与 LLM 解码时段重叠更多，但更不可靠，并可能浪费沙盒容量。关键词路由器行动较早，但必须在假阳性与更严格阈值之间权衡；语义路由器更精确但更慢。要求两者同时预测会引入语义路由器的延迟，而机会主义地组合其输出则可在允许更早决策的同时保持高精度。因此，SpecBox 提出一种意图感知沙盒预热机制，组合这两种异步意图预测，而非认为单独任一者即已足够。

关键词路由器。 关键词路由器对照工具特定的关键词画像扫描数据流，并可在出现辨识性 Token 后的微秒级发出候选，使环境在 LLM 继续解码时完成准备。research、search 或 slide 等常见术语出现在许多工具描述中：仅凭单次匹配触发会产生带有大量假阳性的庞大预热集合，而要求多次匹配则会推迟准备，错过良好的重叠机会。因此，我们对匹配到的工具特定关键词数量施加阈值 γ。在图 3 中，用户请求与演化计划中的表层线索使关键词路由器能够在规划智能体完成本次 ReAct 步骤之前发出可能的沙盒候选。该阈值在早期激活与资源浪费之间取得平衡；所选配置见 § 5.3。

语义路由器。 并行地，语义路由器将活动上下文与工具意图表示进行比较。它能捕捉措辞与工具画像重叠很少的请求，并利用计划与先前生成来消歧通用关键词线索。在图 3 中，即便没有单个 Token 能唯一标识某工具，它也能从更广泛的任务意图中识别候选沙盒。其权衡是时间性的：可靠的语义证据通常需要更长的前缀，因此仅依赖语义的决策往往来得太晚，无法掩盖 PaperSearch（OpenAGS, 2026）与 Neo4j（Neo4j, 2026）等重量级沙盒的冷启动延迟。语义路由器是候选的异步、互补来源，可恢复关键词路由器错过的意图。

并集组装。 令 𝒮key(N) 与 𝒮semantic(N) 为两个路由器在 Token 步骤 N 的候选集。图 4 左下象限说明了为何仅预热其交集并不合适：它提升了表面精度，却使每次触发都等待语义路由器，并且只要任一路由器召回不完善就会丢弃有效工具。相反，我们使用右下策略：

每个路由器各自管理自身的假阳性，并集则让第一个可信信号即可启动准备。图 3 展示了所得行为：路由器从请求与部分计划独立生成候选，再形成统一的预热集合交给沙盒管理器。显式意图受益于关键词路由器的早期预热，而隐式意图由语义路由器处理。如 § 5.3 将展示的，异步并集可最小化等待时间，并在所评估的路由工作负载中消除冷启动。

图 6. 使用一阶马尔可夫模型的随机沙盒预取。左：基于 SDG 的马尔可夫状态转移图，边概率来自观测计数。右：示例会话，展示跨轮次的阈值化、有预算预取，其中预测后继在下一步提交之前被预热。

3.2 随机沙盒预取

意图感知沙盒预热只能利用当前步骤剩余的解码时间；当非驻留沙盒的启动时间长于边际 Token 生成窗口时，这并不充分。实际上，智能体工作流中的步骤虽非确定性，但通常遵循概率模型。例如，论文检索之后，智能体可能阅读返回的文档；数据分析之后，它可能生成图表或报告。SpecBox 利用这些跨步概率模式，在 Tsandbox_exec(N) 期间、下一步提交到具体调用之前，提前导航环境准备。

随机马尔可夫过程建模。 令 𝒱 表示沙盒状态空间，其中每个状态是沙盒类型，或状态级 SDG 中的带类型工具—状态元组。对于执行轨迹 {S(1),…,S(N)}，SpecBox 在有向沙盒依赖图（SDG）中记录步到步的转移。对每个有序对 (vi,vj)，我们维护转移计数：

下一状态概率由带拉普拉斯平滑（Laplace, 1812）的一阶马尔可夫模型估计：

其中 α=1，避免稀疏观测转移出现零概率坍缩。

预算约束下的预取。 给定当前状态 vi，SpecBox 首先识别具有不可忽略冷启动成本的沙盒候选：

其中 Lj 表示沙盒 vj 的估计冷启动惩罚，λ 是轻量成本阈值。

随后 SpecBox 过滤低置信后继，仅保留高概率候选：

其中 τ 控制假阳性预热。最后，SpecBox 在固定预算下选择 top-B 沙盒：

其中 𝒞i 是过滤后的候选集，𝒜i 是最终有预算的预取集，B 限制每步预热的沙盒数量。该策略有意保持轻量，可在 LLM 生成关键路径之外执行。

在线更新。 每次沙盒调用完成后，SpecBox 向 SDG 追加一条转移边，并在后台预取工作线程中异步更新 Ci,j 与 Pi,j。因此，预测器可持续适应演化中的多轮工作流，而不阻塞前台智能体执行。图 5 展示了随机沙盒预取如何与意图感知沙盒预热在完整多步工作流中协同运作。步骤 N 在沙盒 vi 中结束后，SpecBox 查询 SDG 以获得可能的后继沙盒，使用 Lj≥λ 与 Pi,j≥τ 过滤低价值或低置信候选，然后在步骤 N+1 完全提交之前选择有预算的预取集 𝒜i。

图 7. SpecBox 中的复用感知数据传输。语义缓存命中则返回先前结果（paper_search）；未命中则执行沙盒（paper_slides），并通过带外数据路径交付结果。

3.3 复用感知数据传输

一旦调用就绪，其剩余关键路径成本便落在产物移动（Tdata_io(N)）与工具执行（Tsandbox_exec(N)）上。SpecBox 首先检查能否复用缓存结果；否则运行工具，并通过独立数据路径返回其产物。图 7 详述了该过程。

语义缓存。 多轮智能体常常重新访问未变化的文档、以不同措辞提交语义等价查询，或重跑确定性计算。精确参数匹配稳健，但无法捕捉这类表层变化；而不受约束的语义匹配可能错误合并在工具、输入或副作用上不同的请求。因此，缓存必须扩大可复用请求集合，同时严格保持功能等价与行为兼容。

对每次完成的确定性调用，SpecBox 存储规范化调用签名及其结果。查找时，它首先按工具身份过滤缓存条目，再将规范化调用表示与缓存签名比较。对于请求 x，仅当存在满足语义等价约束的兼容缓存条目时，才允许结果复用：

其中 x 表示传入的工具调用请求，xi 表示第 i 条缓存调用。函数 ϕ(·) 将调用变换为规范化语义表示：先去除参数表示中的表层变化，再提取语义特征用于相似性比较。函数 sim(·,·) 度量两个调用表示之间的相似性，阈值 τc 控制语义复用的严格程度。工具身份约束保证接口级兼容，而相似性阈值将复用限制在规范化语义足够相似的调用上。

这一设计将语义匹配视为精确复用的保守扩展，而非不受约束的近似机制。语义相似性仅在同一工具接口与确定性调用行为的边界内扩大可复用请求空间。缓存未命中时，执行沿标准路径进行，并将完整计算结果追加到缓存。缓存命中时，沙盒初始化与工具调用都被跳过，从而降低 Tenv_prep(N) 与 Tsandbox_exec(N)。如 § 5.3.3 将展示的，语义缓存策略可比单独使用精确匹配恢复更大比例的冗余计算，同时维持保守回退路径以保持推理正确性。

带外数据传输。 即便沙盒已完全初始化，传统 RPC 传输仍会将大型日志、文件、图像与结构化输出序列化进请求—响应消息。因此，Tdata_io(N) 随载荷大小增长，并在后续推理步骤之前阻塞智能体。整体替换控制协议会损害与现有 MCP 工具的兼容性；因此，SpecBox 转而将控制信令与产物传输解耦。

带内控制面继续承载标准工具指令、完成通知、错误报告与紧凑元数据。大型产物在该平面上仅以固定大小的引用表示，并通过共置的零拷贝数据路径交换。在此设计下，缓存未命中按常执行、恰好发布一次产物，并向智能体引擎返回引用；缓存命中则简单地复用并经同一数据路径返回先前已发布的产物。这一架构解耦在保持现有控制接口的同时，消除了高容量输出在 RPC 层的序列化与内存拷贝。如 § 5.3.4 所示，相应传输延迟实际上对载荷大小不敏感。

4 实现

图 8. SpecBox 概览。

SpecBox 实现为位于智能体引擎与外部执行沙盒之间的预测性服务运行时。图 8 展示了其控制路径与数据路径。该实现对智能体运行循环进行观测，而不改变其面向工具的接口，并将运行时工作与已经在进行的 LLM 及沙盒工作重叠。

控制面。 在控制面上，控制器订阅传入的 Token 流并记录已完成的工具转移。关键词路由器对照工具关键词画像匹配数据流，当匹配到的工具特定关键词数量达到 γ=2（第 5.3 节选定的工作点）时发出候选。并行地，语义路由器使用该节选定的稀疏检索配置。并集组装将其候选集独立分派给沙盒管理器，后者对请求去重并启动相应容器。后台预取工作线程根据执行轨迹更新沙盒依赖图，应用第 3.2 节的转移概率，并在当前工具执行期间提交有预算的下一步预热。在我们的实现中，成本阈值配置为 λ=5，预取概率阈值为 τ=0.6，每步预取预算为 B=1。

数据路径。 在调度确定性调用之前，运行时构造其调用表示，将查找限制在同一工具身份，并检查语义结果缓存。每条缓存条目包含工具标识符、规范化调用签名、语义嵌入与结果引用。仅当工具身份匹配且语义相似性超过报告实验中使用的阈值 τc=0.8 时，缓存才复用先前结果。未命中时，沙盒将大型结果写入主机管理的内存映射共享内存区域。控制面仅携带命名该区域的 64 位 token_id，而智能体引擎直接从共享内存底板读取结果。小型指令、元数据、完成通知与错误仍走普通控制面。这实现了图 7 中常见的“缓存或执行”流程，而无需通过 RPC 栈序列化大宗结果。

执行边界与正确性。 我们考虑多租户智能体，其工具运行在操作系统隔离的容器或微虚拟机中。预测性准备使环境就绪，但在智能体提交调用之前并不执行具有副作用的工作；未使用的预热会被丢弃。缓存复用限于确定性、兼容的工具请求。共享内存桥接运行在单一受信任主机或安全托管的集群边界内，现有访问控制防止跨租户内存访问。沙盒逃逸、恶意智能体与被攻破的基础设施不在本工作的威胁模型之内。

5 评估

5.1 实验设置

硬件与软件环境。 全部实验在一台商用服务器上进行，配备 16 核 CPU、256 GiB 主机内存与 2 TB NVMe SSD。SpecBox 构建于 AgentScope（Gao et al., 2024）框架之上，具体集成于其智能体引擎层。隔离的多租户执行沙盒通过 Docker 容器实例化，整个运行时基础设施以 Python 实现。为驱动智能体推理，我们使用阿里云百炼的 Qwen3.5-Max 模型（Alibaba Cloud, 2026），通过其生产云 API 端点并发访问。

工作负载。 我们使用轨迹级基准评估 SpecBox，包含 200 条多轮轨迹，源自 MCPBench（Wang et al., 2025），涵盖从 GitHub（GitHub, 2026）收集的 32 个开源 MCP 兼容工具服务器（例如 Playwright（Microsoft, 2026b）、Jupyter（Datalayer, 2026）、Neo4j（Neo4j, 2026））。为确保工具级有效性与现实的工作流执行依赖，我们分两阶段构建该基准：

  • 原子工具使用生成： 我们为每个工具生成 20 个单轮交互模板（共 640 个），以覆盖全部 32 个工具的多样化原子行为。

  • 多轮轨迹构建： LLM 规划器增量生成 1–10 步的智能体会话，其中每个规划任务都针对真实服务器执行，并将真实执行结果回馈到后续规划步骤。

所得数据集按轨迹长度均匀分布，每种长度 20 条轨迹，平均每会话 6.4 步、每步 2.96 个工具。这种以执行为根基的方法确保了工具级有效性，消除了工具特定偏差，同时反映了有代表性的 LLM 智能体工作流。

方法。 除非另有说明，我们以固定随机种子（seed = 0）将完整轨迹作为会话级工作负载回放，以评估所有系统。每条轨迹逐步执行，保留推理、工具调用与沙盒执行之间的依赖，以忠实建模现实的多轮智能体工作流。在这一确定性采样设置下，所得轨迹分布自然集中在 5–8 步区间，因此我们将其视为代表性工作负载区间，而非人工选定的子集。

除非明确另行说明，我们在会话级评估全部实验，并将工作负载特定配置（如轨迹长度或采样范围）留待各消融研究自行说明。

图 9. 多轮智能体会话（每会话 5–8 步）的累积沙盒供给延迟。

图 10. 并发工作负载下的端到端性能与资源消耗：(a) P99 端到端延迟，(b) 平均累积沙盒供给延迟，(c) 峰值 CPU 使用，(d) 峰值内存使用。

指标。 我们从与系统设计目标对齐的三个维度评估 SpecBox：

  • 端到端延迟： 对每个智能体会话，我们报告平均延迟与尾延迟（P99），以捕捉多步执行下的平均性能与长尾行为。

  • 资源效率： 我们在多租户工作负载下测量峰值 CPU 与内存消耗，以量化沙盒供给、执行与生命周期管理引入的运行时开销。

  • 预测精度： 我们评估全部预测性运行时优化的精度，包括意图感知沙盒预热、随机沙盒预取与语义缓存。对每一项，我们报告 i）在不确定工具意图下正确的触发或检索决策，以及 ii）假阳性率，反映不必要或不正确的激活，如误路由意图、冗余预热。

5.2 端到端性能

本节在智能体服务的两个代表性基线下评估 SpecBox 的端到端（E2E）性能。基线定义如下：

  • 预留运行时： 为所有候选工具维持永久预热沙盒。这代表延迟下界（性能上限）。然而，由于在大规模、多租户 MCP 工具生态系统中空闲内存占用过高，它在生产中财务上与物理上均不可行。

  • 按需运行时： 在工具调用时动态实例化沙盒，镜像生产 MCP 运行时（Amazon Web Services, 2026；Microsoft, 2026a；Google Cloud, 2026）。由于其瓶颈源于应用级工具绑定与会话握手，而非通用操作系统启动，它仍是标准的实用基线。

我们旨在回答一个根本问题：SpecBox 能否成功将执行延迟与物理资源约束解耦，以接近有服务器的性能、接近无服务器的成本？

5.2.1 多轮智能体工作流中的累积沙盒供给延迟

我们评估 SpecBox 对多轮智能体执行轨迹上沙盒供给延迟的端到端影响。我们将累积沙盒供给延迟定义为会话内每次工具调用所产生的沙盒初始化延迟之和，仅关注环境建立开销，排除沙盒内计算时间。

图 9 展示了跨会话的累积延迟分布。按需基线由于反复的冷启动开销，在更长执行视野上延迟稳步增加，形成重尾分布。相比之下，SpecBox 显著降低累积延迟，并保持集中在亚秒区间的更紧分布。相对 按需 基线，SpecBox 将累积沙盒供给延迟降低 4.53×。相对 预留 基线，SpecBox 的性能差距保持在 10.6% 以内，同时避免了持久沙盒分配的巨大资源开销。这些结果表明，预测性编排能在 按需 部署模型下有效逼近 预留 执行性能。

5.2.2 可扩展性

图 10(a) 总结了三种运行时在不同并发下的可扩展性，QPS 从 1 增至 20。在低到中等并发水平，SpecBox 始终保持接近 预留 的延迟曲线，同时相对 按需 降低端到端延迟。在更高并发下，相对 按需 的优势依然显著：在 QPS=20 时，Laplace 达到 88.7s 的 P99 端到端延迟，相对 按需（257.2s）加速 2.9×。这表明 SpecBox 能够吸收不断增长的并发，而不会继承 按需 基线的长尾延迟爆炸。

图 10(b) 还揭示，按需 的主要可扩展性瓶颈在于累积沙盒供给。在低 QPS 区间，三种模式的平均累积沙盒供给延迟均低于 4 秒。然而，一旦负载进入更高 QPS 区间（QPS≥5），按需 因网络争用与资源约束而持续劣化，其累积沙盒供给延迟稳步上升并最终超过 50 秒。相比之下，Laplace 被严格限制在 5 秒以下区间，并在供给延迟上继续略优于 预留，反映出 SpecBox 的意图感知沙盒预热与随机沙盒预取在并发压力下的收益。

5.2.3 资源占用与效率

图 10(c) 显示，三种运行时的 CPU 占用在低负载下相对紧凑，但随并发扩展而显著分化。虽然 SpecBox 在低 QPS 负载下没有明显优势，其长处在 QPS≥8 的高 QPS 区间逐渐显现。它在所有测试负载下持续抑制峰值 CPU 利用率，峰值资源消耗约封顶在 12.2 核。相对 按需 与 预留（约需 15.8–15.9 核），这相当于峰值 CPU 压力降低 22.8%–23.3%。尤其在高 QPS 区间，SpecBox 稳定在约 11–12 核的紧区间，而两个基线在 14–16 核的更高平台上波动，表明 SpecBox 有效缓解了 CPU 争用（峰值负载下最多节省 25% CPU 资源），同时不牺牲亚毫秒级控制面响应性。

内存占用（图 10(d)）在评估谱上呈现出显著的资源分离。如预期，预留 因其持久化策略是内存最密集的变体；其内存消耗从轻负载时的初始 24.6 GiB 飙升至高达 80.6 GiB 的峰值，并在并发加剧时持续高于 60 GiB。相反，按需 保持精简画像，峰值使用介于 24.3 GiB 与 40.4 GiB 之间。值得注意的是，SpecBox 在大多数 QPS 配置下紧密镜像 按需 的轨迹，最高达 49.4 GiB。相对 预留，这代表峰值内存占用降低 45.9%，表明 SpecBox 成功消除了长运行沙盒过高的内存占用成本，同时在重度并发工作流下保持可预测的、类无服务器的资源弹性。

5.3 微基准测试

本节为上文报告的端到端改善提供机制级归因。具体而言，我们隔离意图感知沙盒预热、随机沙盒预取、语义缓存与带外数据传输的效果，以避免在 E2E 部分混淆各组件贡献。

5.3.1 意图感知沙盒预热的有效性

我们进一步将意图感知沙盒预热分解为三个正交因素：关键词触发阈值、语义模型族，以及混合融合策略。全部结果在相同采样任务上测量，并重复 100 次。

关键词路由器阈值 γ 的敏感性。

为严格隔离关键词匹配阈值的影响，我们在相同工作负载画像下评估 γ∈{1,2,3}，每种配置执行 100 次独立试验以确保统计收敛。如表 1 所量化，γ=2 成为关键词路由器的最优配置。具体而言，γ=2 将关键词驱动的平均等待延迟大幅压缩至 323.0 ms（相对 γ=1 下的 786.6 ms 降低 2.43×），同时保持 95% 的高预测路由覆盖率。

γ=1 下的严重延迟劣化主要由其超敏感性驱动，在 100 次运行中触发 20 次不同的工具不匹配；这些假阳性无意中放大了冷启动回退惩罚，并引入重尾延迟。相反，将阈值提高到 γ=3 进一步锐化路由精度（匹配率 97%，仅 3 次不匹配），但严重侵蚀预热领先时间，使平均等待时间回升至 621.8 ms。这一经验权衡确认，γ=2 有效平衡了早期预测敏捷性与资源稳定性。

表 1. 关键词阈值敏感性（γ=1,2,3）。

γ 平均等待 (ms) 匹配率 不匹配运行次数
1 786.619 80.0% 20
2 323.021 95.0% 5
3 621.812 97.0% 3
语义路由器模型的敏感性

表 2 比较了三种语义路由模型在各自最优配置下的性能与计算权衡：

  • Retrieval： 该模型实现为非神经、稀疏的 Token 级检索器。它利用 TF-IDF（Sparck Jones, 1972）加权，结合 Top-N 近邻聚合网络，跨 Token 一元组、选择性二元组与字符 n-gram（3 到 5 个字符）构造复合向量特征。

  • Encoder： 该神经模型基于微调的 all-MiniLM-L6-v2（sentence-transformers, 2026）Transformer 序列表示网络。它通过 CosineSimilarityLoss 约束下的对比成对优化，将流动的多轮轨迹映射到稠密向量空间。

  • FastText： 该轻量模型（Joulin et al., 2017）将文本表示为词与子词嵌入的平均，后接线性分类器。

经验上，如表 2 所量化，Retrieval 与 Encoder 模型均达到最优预热命中率，表明稀疏词汇 Token 与稠密语义特征都能成功捕捉全部必要的工具调用目标。然而，Retrieval 获得更高的编排质量与路由精度，Micro-F1 为 0.970、Precision 为 0.942，而 Encoder 降至 Micro-F1 0.776、Precision 0.634。

此外，从运行时效率角度看，Retrieval 的计算占用显著更低，平均推理延迟仅 2.116 ms——相对 Encoder 的 7.822 ms 延迟加速 3.70×。相反，虽然 FastText 提供超低执行延迟（0.171 ms），其浅层 Token 表示空间在多轮推理上下文漂移下无法提供可用判别力，Micro-F1 与命中率仅为 0.124。因此，Retrieval 被选为 Laplace 语义分支的生产实例，因为它在高保真预测精度与低开销控制面延迟之间达到最优帕累托效率前沿。

表 2. 语义路由器模型比较（每种模型的最佳工作点）。

模型 Micro-F1 命中率 (Top-3) Precision 平均延迟 (ms)
Retrieval 0.970 0.992 0.942 2.116
Encoder 0.776 0.968 0.634 7.822
FastText 0.124 0.124 0.124 0.171
组装策略的敏感性。

为严格隔离意图组合层的算法影响，我们在相同多轮上下文分布下评估三种不同的融合策略范式：

  • 并集（∪）： 预测性预热原语是非阻塞的，在关键词路由器或语义路由器命中各自激活边界的精确微秒时刻异步分派。

  • 交集（∩）： 预测性编排严格强制双路由器共识；当且仅当关键词路由器与语义路由器同时验证该沙盒候选的调用意图时，才预热沙盒。

  • 加权： 该混合策略将多模态意图度量聚合为中心化候选分数：s(c)=wk·sk(c)+ws·ss(c)，其中 sk(c) 表示匹配关键词比率（相对静态词表规模归一化），ss(c) 表示实时语义检索相似性向量。仅当越过刚性阈值 s(c)≥τf 时才分派预测触发。在我们的经验运行器中，我们实现对称基线配置：wk=0.5，ws=0.5，τf=0.5。

如表 3 所量化，激进的 并集组装 与保守的 交集 策略之间存在清晰权衡。并集组装 优先延迟掩盖敏捷性，将平均等待延迟降至仅 124.45 ms，代价是轻微的 5.0% 冷启动比率。相反，保守的 交集 策略通过强制严格双路由器共识消除冷启动（0.0%），但代价是阻塞关键路径，并将平均延迟推高至 393.52 ms（比并集高 3.16×）。

加权 策略表现最差，因 45.0% 的冷启动比率将延迟膨胀至 1308.38 ms。这一失败源于关键词 Token 与高维语义之间的结构错位。在没有动态重归一化的情况下，多轮推理下移动的上下文分布使组合分数漂移，频繁无法越过静态 τf=0.5 阈值。这些结果验证了我们选择 并集优先 组装设计，以最大化延迟掩盖性能，同时保持实用的路由正确性。

表 3. 组装策略权衡。

组装模式 平均等待 (ms) 目标匹配率 冷启动比率
Union 124.446 95.0% 5.0%
Intersection 393.523 100.0% 0.0%
Weighted 1308.378 55.0% 45.0%

5.3.2 随机沙盒预取的有效性

图 11. 跨 10 轮对话视野的动态执行剖析，对比每轮平均等待延迟（左）与对应的平均冷启动沙盒激活次数（右）。

为在多轮智能体执行下隔离随机预取的贡献，我们在相同规划工作负载下评估 Laplace 的两种部署变体：

  • SpecBox-Reactive： 仅路由的基线，执行内联 Token 级意图检测，但不使用跨步转移预测。

  • SpecBox-Proactive： 完整设计，将在线混合路由与沙盒依赖图（SDG）上的随机马尔可夫预取器耦合，使下一步之前即可异步准备沙盒。

图 11 报告 10 轮视野上的每轮平均等待延迟与冷启动次数。在第 1 轮，两种变体延迟相近（512.06 ms vs. 540.06 ms），因为尚无历史转移信号可用于初始化马尔可夫预测器。从第 2 轮起，两条轨迹急剧分化。SpecBox-Proactive 将平均等待延迟从第 2 轮的 540.06 ms 降至 138.14 ms，并在第 10 轮进一步降至 97.14 ms，而 SpecBox-Reactive 在第 10 轮升至 583.26 ms。这对应 6.0× 的视野末端延迟降低。该机制与冷启动遥测一致。在 SpecBox-Reactive 下，平均冷启动次数在第 9 轮达到每轮 2.90 次峰值，表明关键路径上反复暴露于沙盒初始化成本。相反，SpecBox-Proactive 将每轮冷启动保持在 0.24–0.83，通过早期调度有效掩盖启动延迟。这些结果表明，Laplace 通过准确、低开销的时间预取而非基础设施过度供给，改善了多轮响应性。

5.3.3 语义缓存的有效性

为量化语义缓存的影响，我们在相同重复请求工作负载上、100 次运行中基准测试三种变体：

  • 无缓存： 每次重复请求都从零执行。

  • 精确匹配缓存： 严格缓存，按工具身份与规范化参数相等进行匹配。

  • 语义缓存： 当工具身份匹配且语义相似性超过阈值时复用结果。

表 4. 语义缓存权衡。

设置 平均等待 (ms) 命中率 旁路比率
无缓存 412.78 0.0% 0.0%
精确匹配缓存 233.64 33.6% 100.0%
语义缓存 (τ=0.6) 141.93 37.4% 84.8%

如表 4 所示，虽然精确匹配缓存相对 无缓存 改善了性能，语义缓存通过容忍表层形式变化捕捉了更宽的近重复请求包络。相对 无缓存，语义缓存将平均等待延迟加速 2.91×（从 412.78 ms 降至 141.93 ms），并将缓存命中率提升至 37.4%。

然而，语义缓存的旁路比率略低于精确匹配，因为一小部分语义命中尚不足够可靠以跳过沙盒建立，必须在验证后回退到正常执行。这一行为与预期设计一致：语义等价扩大可复用请求空间，但某些宽松匹配在旁路执行之前仍需保守验证。

5.3.4 带外数据传输的有效性

图 12. 不同载荷大小下的数据传输延迟扩展画像，比较 SpecBox 的带外数据传输与标准 JSON-RPC 序列化。

为评估控制面与数据面分离的架构效率，我们通过在从 1.00 MB 到 1000.00 MB 指数扩展的数据载荷谱上基准测试两种不同传输范式，隔离数据传输开销：

  • 带外： 我们提出的带外传输机制。它通过将大宗状态载荷直接写入专用本地共享内存基底或零拷贝虚拟化主机—客户机环形缓冲区，完全绕过控制面 RPC 隧道，仅在线路上传递轻量、固定大小的引用。

  • JSON-RPC： 传统智能体运行时使用的标准基线范式。它将多模态数据载荷直接编组到内联运行时执行流中，迫使控制面通过基于文本的 JSON-RPC 网络原语同步序列化并传输原始数据矩阵。

如图 12 所示，经验测量揭示了两种协议在指数载荷谱上的清晰扩展分化。在基线阈值（1.00 MB），两种配置均展现亚 3 毫秒性能，带外 保持轻微优势（1.95 ms vs. 2.28 ms）。然而，随着载荷扩大，JSON-RPC 流水线遭受灾难性的线性性能退化；受沉重序列化瓶颈节流，其延迟在 100.00 MB 增至 132.45 ms，并在 1000.00 MB 边界达到 1873.16 ms。相反，带外 展现近恒定的 O(1) 扩展画像，在 1 GB 边界仅漂移至 5.97 ms——延迟降低 313.55×。

这一性能差距源于消除了关键路径上的序列化与内存拷贝。在多轮工作流中，智能体频繁交换大型多模态状态，如高维向量或媒体文件。标准 JSON-RPC 因同步序列化而使控制循环停滞，而 SpecBox 的带外设计将控制信号与原始载荷路由解耦，将数据传输简化为常数时间的引用传递操作。这些结果表明，分离控制面与数据面使 SpecBox 的编排开销保持最小，且与载荷大小无关。

6 讨论

架构开销。 预测性服务运行时的一个关键关切是：系统级编排机制是否会在关键路径上引入不可忽视的处理惩罚。在 SpecBox 中，这一控制面开销通过严格的带外执行与并行路由机制，与 GPU 绑定的推理循环彻底隔离。Token 级扫描以确定性 𝒪(1) 复杂度运行，而固有更重的语义检索引擎被卸载到专用后台 CPU 工作线程。由于意图路由器采用 并集组装（∪），关键路径从不等待迟到的语义嵌入；任何早期快速路径触发都会在微秒内立即分派激活原语。结合预取守护进程——它评估由活跃沙盒小基数所界定的低维马尔可夫矩阵转移——SpecBox 将其控制面遥测限制在数十微秒，从而对 Token 生成实现近乎零成本的延迟影响。

生态泛化。 我们将 SpecBox 定位为运行在上游推理智能体与下游执行环境之间的运行时中间件层。这一设计利用现代智能体生态中两项广泛可用的能力（Wu et al., 2023；Langchain-ai, 2026；Gao et al., 2024）：上游框架的流式 Token 级输出，以及 MCP（Anthropic PBC, 2024）等协议提供的标准化工具接口。因此，SpecBox 除现有智能体与沙盒编排栈已支持的能力外，不要求额外的系统级约束。

重要的是，这一执行抽象可从推理时服务延伸到智能体强化学习（Agentic RL）框架，如 VERL（Sheng et al., 2024）与 Slime（Zhu et al., 2025），其中 rollout 生成与环境交互遵循结构相似的循环。在这些 Agentic RL 设置中，SpecBox 的运行时优化——即 意图感知沙盒预热 与 随机沙盒预取——可以较小的集成代价无缝适配，以在训练 rollout 期间掩盖环境初始化与交互开销。

局限与未来工作。 尽管取得了性能收益，SpecBox 仍表现出某些架构边界。在控制面上，我们的预取器经由马尔可夫链模型假设一阶历史依赖，这在开放式、长时程智能体工作流中可能遭受预测精度下降。关键的是，SpecBox 通过其分层、两级准备内在地缓解了这一点：任何跨步预取未命中都会优雅回退到实时 Token 流期间的步内意图感知预热。这一设计保证最坏情况下的环境延迟被严格限制在 LLM 解码阶段之内。未来工作将研究在线图神经网络（GNN），以自适应捕捉非线性转移模式。

在数据面上，我们基于 mmap 的传输要求将智能体引擎与沙盒共置在同一主机边界内。虽然这一设计与主流多租户服务拓扑无缝对齐（例如 Kubernetes Pod IPC 共享或 Sidecar 模式），并利用稳健的容器级命名空间与 cgroup 安全隔离，它限制了单会话跨节点扩展。为支持大规模分布式集群，我们计划集成 RDMA 辅助的零拷贝传输，将带外数据面延伸到解耦的云基础设施中。

7 相关工作

轻量沙盒运行时。 系统社区已积极探索轻量隔离机制，以缓解无服务器执行环境的物理实例化成本。显著进展包括 Firecracker（Agache et al., 2020）与 RunD（Li et al., 2022）等微虚拟机（MicroVM）、WebAssembly（Wasm）（WebAssembly, 2026）运行时，以及 FaaSnap（Ao et al., 2022）与 TrEnv-X（Huang et al., 2026）等进程级快照/分叉框架。这些基础设施级系统聚焦于最小化局部主机建立，或利用硬件辅助的远程内存池（如 CXL/RDMA）跨租户共享并复用物理沙盒。

关键的是，这些低层容器优化与 SpecBox 完全正交。虽然微虚拟机快照或可再利用沙盒成功将物理基础设施启动时间压缩到毫秒级，应用层握手（如 MCP 发现钩子）与环境附着瓶颈仍会原生滞留在关键运行时路径上。SpecBox 运行在更高的、应用感知的编排层；它通过利用一个不同的时间维度——将控制面调度与流式 Token 生成重叠——来补充这些物理加速，从而完全掩盖而非压缩固有的就绪延迟。

无服务器预测性预热。 预测性预热是标准无服务器框架中被广泛采用的技术，用以消除租户冷启动臭名昭著的尾延迟惩罚。最先进的预取守护进程（如 Mitosis（Wei et al., 2023）与 IceBreaker（Roy et al., 2022））主要依赖历史时间序列分析、统计调用频率直方图，或时间相关聚类，以预测性地准备空闲容器运行时。

然而，这些经典预热范式内在假设传入请求遵循独立同分布到达模型或确定性时间触发模式。这一核心假设在新兴 LLM 智能体工作负载下彻底失效。在多轮自主智能体推理循环中，工具调用由 LLM 流动的上下文轨迹动态、自主且非线性地决定，呈现出复杂的执行依赖视野。SpecBox 通过将服务基础设施与显式的智能体行为特征协同设计来弥合这一鸿沟，引入构建在沙盒依赖图（SDG）之上的随机马尔可夫预测框架，以建模实时自主状态转移。

LLM 服务与智能体编排。 加速大语言模型的端到端执行，推动了 AI 系统谱系上的广泛研究。在模型服务边界，vLLM（Kwon et al., 2023）与 SGLang（Zheng et al., 2024）等主流运行时优化 GPU 内部内核执行、经由 PagedAttention 的内存缓存，以及自动化推测解码。与此并行，AgentScope（Gao et al., 2024）、AutoGen（Wu et al., 2023）与 LangGraph（Langchain-ai, 2026）等应用级智能体编排平台，为编程多轮自主多智能体团队提供模块化抽象。

然而，现有 LLM 服务引擎从根本上将模型执行视为自包含的 GPU 计算单元，完全无视与外部工具交互时物理主机平面的环境摩擦。相反，高层智能体框架缺乏对云原生主机拓扑的系统级可见性，导致不协调的数据传输与执行停滞。SpecBox 作为运行时中间件层运作以弥合这一鸿沟，在异构多租户主机执行边界上实现协同优化的带外控制面预测路由与共享内存数据传输。

8 结论

我们提出 SpecBox，一个面向基于 LLM 的智能体系统的预测性执行运行时，用于降低多租户环境中的尾延迟与资源低效。通过分析端到端智能体执行步骤，我们发现低效主要源于推理、环境初始化与沙盒执行之间的僵硬依赖。SpecBox 通过统一设计加以应对：使 LLM 执行与沙盒建立在时间上重叠，跨步骤预判未来沙盒需求以降低跨步冷启动开销，并通过复用感知执行与带外数据传输消除冗余计算与通信。SpecBox 实现于 AgentScope（Gao et al., 2024）之上，并在高并发多轮工作负载下进行评估。结果显示 P99 延迟最多降低 2.9×，峰值内存使用降低 45.9%，预热命中率为 97.9%。这些结果表明，利用智能体执行循环中的时间重叠，是改善大规模智能体部署效率的有效且通用的机制。

3.4 - 论文中英文双语

论文的中英文双语对照版本

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

SpecBox：面向高效 LLM 智能体服务的推测性沙盒调度

Abstract / 摘要

As LLM agents increasingly rely on the Model Context Protocol (MCP) to invoke isolated external sandboxes, disaggregated sandbox deployment introduces a fundamental tension between resource utilization and interactive tail latency. Persistent long-lived sandbox reservations incur excessive memory overhead at scale, while lazy on-demand instantiation generates severe cold-start penalties that degrade response performance under multi-tenant, multi-turn agent workloads. To resolve this dilemma, we present SpecBox, a runtime built around speculative sandbox preallocation tailored for dynamic LLM agent execution pipelines.

随着 LLM 智能体日益依赖模型上下文协议（Model Context Protocol, MCP）来调用隔离的外部沙盒，解耦式沙盒部署在资源利用率与交互式尾延迟之间引入了根本张力。持久的长生命周期沙盒预留会在规模化时带来过高的内存开销，而惰性按需实例化又会产生严重的冷启动惩罚，从而在多租户、多轮智能体工作负载下劣化响应性能。为化解这一两难，我们提出 SpecBox——一个围绕推测性沙盒预分配构建的运行时，专为动态 LLM 智能体执行流水线设计。

At its core, SpecBox implements keyword matching and streaming semantic embedding to enable intent-driven sandbox prewarming, which identifies pending tool execution demands mid-LLM token generation and fully overlaps sandbox bootstrapping with model inference. To extend prewarming windows across sequential agent steps, the framework leverages context-aware stochastic prefetching atop a sandbox dependency graph to probabilistically forecast future sandbox switches ahead of execution. We complement these speculative mechanisms with two orthogonal optimizations: a semantic result cache that prunes redundant repeated sandbox invocations, and a dedicated out-of-band shared-memory transport plane that bypasses conventional network serialization to deliver zero-copy artifact transfers. Evaluated on high-concurrency multi-turn agent traces, our prototype demonstrates that SpecBox cuts P99 end-to-end latency by up to 2.9× relative to the on-demand sandbox baseline, while slashing peak memory consumption by 45.9% compared to permanently reserved sandbox deployments.

其核心是：SpecBox 实现关键词匹配与流式语义嵌入，以支持意图驱动的沙盒预热；该机制在 LLM Token 生成过程中识别即将到来的工具执行需求，并将沙盒启动与模型推理完全重叠。为将预热窗口延伸到后续智能体步骤，框架在沙盒依赖图之上利用上下文感知的随机预取，在执行之前以概率方式预测未来的沙盒切换。我们还辅以两项正交优化：语义结果缓存，用于裁剪冗余的重复沙盒调用；以及专用的带外共享内存传输平面，绕过传统网络序列化，实现产物的零拷贝传输。在高并发多轮智能体轨迹上的评估表明，相对按需沙盒基线，SpecBox 可将 P99 端到端延迟最多降低 2.9×，同时相对永久预留沙盒部署，峰值内存消耗下降 45.9%。

Keywords / 关键词: LLM Agent, Execution Runtime, Predictive Prewarm / LLM 智能体、执行运行时、预测性预热

1 Introduction / 引言

Modern LLM agents are evolving from conventional text generators into autonomous systems that iteratively reason, plan, and execute external actions (Yao et al., 2022; Li et al., 2024b; Wang et al., 2024). Unlike traditional LLM serving workloads that primarily optimize token generation (Agrawal et al., 2024; Zhang et al., 2025), agent execution forms a stateful run-loop where an LLM-based controller repeatedly invokes external tools (Microsoft, 2026b; GitHub, 2026; Datalayer, 2026; Neo4j, 2026) such as code execution, web automation, and data processing services. For security, isolation, and reproducibility, these tools are increasingly deployed as independent sandbox environments and accessed through standardized interfaces such as the Model Context Protocol (MCP) (Anthropic PBC, 2024). Consequently, the latency of agent execution is no longer determined solely by model inference, but also by the efficiency of coordinating heterogeneous external execution environments.

现代 LLM 智能体正从传统的文本生成器演进为可迭代推理、规划并执行外部动作的自主系统（Yao et al., 2022；Li et al., 2024b；Wang et al., 2024）。与主要优化 Token 生成的传统 LLM 服务工作负载不同（Agrawal et al., 2024；Zhang et al., 2025），智能体执行形成有状态的运行循环：基于 LLM 的控制器反复调用外部工具（Microsoft, 2026b；GitHub, 2026；Datalayer, 2026；Neo4j, 2026），例如代码执行、网页自动化与数据处理服务。出于安全、隔离与可复现性，这些工具正越来越多地部署为独立沙盒环境，并通过模型上下文协议（MCP）（Anthropic PBC, 2024）等标准化接口访问。因此，智能体执行的延迟不再仅由模型推理决定，还取决于协调异构外部执行环境的效率。

Modern cloud-native infrastructures increasingly adopt serverless sandbox execution to support large-scale concurrent agent sessions (Amazon Web Services, 2026; Google Cloud, 2026; Microsoft, 2026a), i.e., the execution environments are initialized on demand rather than being reserved as long-lived instances. This design enables the required elasticity for multi-tenant agent workloads but introduces non-negligible latency (Puliafito et al., 2022; Yu et al., 2024; Xu et al., 2024; Stojkovic et al., 2023a). Each tool invocation may incur sandbox initialization overhead — such as image loading, filesystem preparation, namespace configuration, and runtime handshake — introducing a second-level delay before execution begins (Huang et al., 2026). In multi-turn agent workflows, such startup overheads accumulate across successive tool invocations, significantly degrading end-to-end responsiveness and limiting the practicality of interactive agent services.

现代云原生基础设施日益采用无服务器沙盒执行，以支撑大规模并发智能体会话（Amazon Web Services, 2026；Google Cloud, 2026；Microsoft, 2026a），即执行环境按需初始化，而非作为长生命周期实例预留。这一设计为多租户智能体工作负载提供了所需弹性，但引入了不可忽视的延迟（Puliafito et al., 2022；Yu et al., 2024；Xu et al., 2024；Stojkovic et al., 2023a）。每次工具调用都可能产生沙盒初始化开销——如镜像加载、文件系统准备、命名空间配置与运行时握手——从而在执行开始前引入秒级延迟（Huang et al., 2026）。在多轮智能体工作流中，这类启动开销会在连续工具调用间累积，显著劣化端到端响应性，并限制交互式智能体服务的实用性。

EN Figure 1: Proactive and overlapped execution v.s. the native vanilla approaches CN 图 1: 主动重叠执行与原生朴素方法的对比。

The main cause of this latency is not sandbox initialization, but the sequential execution model used by the vanilla implementation of the de facto agent runtimes (e.g., AgentScope(Gao et al., 2024), AutoGen(Wu et al., 2023), and LangGraph (Langchain-ai, 2026)). These systems use a reactive and sequential model without pipelined orchestration: the stage of sandbox preparation begins only after the stage of LLM inference finishes the token generation, and the tool invocation is fully determined. As a result, the preparation of the environment remains entirely exposed on the critical path, leaving idle periods between model reasoning and tool execution. While GPUs are occupied generating tokens, CPU-side resources remain underutilized — once sandbox initialization begins, the GPU has to wait for external execution to complete. As illustrated in Fig. 1, the reactive model serializes LLM reasoning and environment preparation, without effectively overlapping computation with environment provisioning.

这一延迟的主要原因并非沙盒初始化本身，而是事实标准智能体运行时（如 AgentScope（Gao et al., 2024）、AutoGen（Wu et al., 2023）与 LangGraph（Langchain-ai, 2026））的朴素实现所采用的串行执行模型。这些系统使用反应式、无流水线编排的串行模型：沙盒准备阶段仅在 LLM 推理完成 Token 生成、工具调用被完全确定之后才开始。因此，环境准备完全暴露在关键路径上，在模型推理与工具执行之间留下空闲时段。当 GPU 忙于生成 Token 时，CPU 侧资源利用不足——一旦沙盒初始化开始，GPU 又必须等待外部执行完成。如图 1 所示，反应式模型将 LLM 推理与环境准备串行化，未能有效地将计算与环境供给重叠。

Existing runtime optimizations such as speculative execution (Stojkovic et al., 2023b; Li et al., 2024a), prewarming (Mahgoub et al., 2022; Sui et al., 2024), workflow orchestration (Liu et al., 2023; Li et al., 2023), and caching (Xu et al., 2024; Lu et al., 2024) are well studied in serverless and cloud systems. However, they do not directly transfer to LLM agent runtimes because agent execution differs fundamentally from conventional request processing. Prior work usually assumes fixed execution targets or pre-defined workflow graph before runtime. In contrast, autonomous agent workflows are produced online via auto-regressive reasoning: tool calls emerge progressively from streaming token generation and remain uncertain until enough semantic context is available. Multi-turn sessions further evolve based on intermediate observations and outcomes, making future tool choices both workflow-dependent and highly dynamic. These characteristics make simple extensions of existing techniques infeasible.

现有运行时优化，如推测执行（Stojkovic et al., 2023b；Li et al., 2024a）、预热（Mahgoub et al., 2022；Sui et al., 2024）、工作流编排（Liu et al., 2023；Li et al., 2023）与缓存（Xu et al., 2024；Lu et al., 2024），在无服务器与云系统中已有充分研究。然而，它们并不能直接迁移到 LLM 智能体运行时，因为智能体执行与传统请求处理存在根本差异。既有工作通常假设固定的执行目标，或在运行前已有预定义工作流图。相反，自主智能体工作流是通过自回归推理在线产生的：工具调用从流式 Token 生成中逐步涌现，直到足够的语义上下文可用之前都保持不确定。多轮会话还会依据中间观测与结果进一步演化，使未来的工具选择既依赖工作流又高度动态。这些特征使得对现有技术的简单扩展并不可行。

In the context of LLM agent runtimes, the key idea is therefore to pre-launch the most likely sandbox before it is requested, overlapping environment preparation with ongoing LLM generation. However, realizing this idea for autonomous agent systems introduces three unique challenges. First, within a single step (agent iteration), intents must be inferred from early token streams with only partial semantics; using short contexts with low thresholds or broad candidate sets improves overlap but risks false-positive prewarming and wasted memory/CPU. Second, across steps, predicting which tool will be invoked in future steps becomes increasingly unreliable as the prediction window extends; prewarming all plausible successors would revert to reserved-deployment costs. Third, prewarming alone removes only sandbox startup delay: repeated tool executions and large artifact transfers can still remain on the critical path even when a sandbox is ready. The system must therefore reuse semantically equivalent execution results and decouple bulk data transfer from control signaling, while preserving protocol compatibility and user-visible semantics.

因此，在 LLM 智能体运行时的语境下，关键思路是在沙盒被请求之前预先启动最可能需要的沙盒，使环境准备与正在进行的 LLM 生成重叠。然而，将这一思路落实到自主智能体系统会带来三项独特挑战。第一，在单步（智能体迭代）内部，意图必须从仅具部分语义的早期 Token 流中推断；使用短上下文、低阈值或宽候选集会改善重叠，但会带来假阳性预热以及内存/CPU 浪费的风险。第二，跨步骤预测未来将调用哪个工具，会随预测窗口拉长而越来越不可靠；若对所有可能后继都预热，成本将退化回预留部署。第三，仅预热只能消除沙盒启动延迟：即便沙盒已就绪，重复的工具执行与大型产物传输仍可能停留在关键路径上。因此，系统必须复用语义等价的执行结果，并将大宗数据传输与控制信令解耦，同时保持协议兼容性与用户可见语义。

This paper presents SpecBox, a predictive serving system that orchestrates tasks of an agent workflow efficiently. SpecBox overlaps LLM execution with environment preparation, breaking the rigid sequential dependencies in vanilla implementation of agent runtimes. SpecBox diminish end-to-end latency through three synergetic techniques: i) intent-aware sandbox prewarming that speculates execution intents on the basis of streaming token outputs, at a proper time, and overlap sandbox preparation with ongoing generation within a step; ii) stochastic sandbox prefetching, leveraging historical agent execution traces to anticipate future sandbox needs and prepares the sandboxes for the next step during the current step, thereby lowering cold-start latency; and iii) reuse-aware data transmission that exploits semantic similarity to bypass redundant tool execution through semantic caching and decouples large artifact delivery from control signaling through an out-of-band data path, eliminating unnecessary computation and serialization overhead. SpecBox’s prototype is implemented and integrated with the open-source AgentScope framework (Gao et al., 2024). SpecBox is framework-agnostic and highly extensible to any LLM agent serving infrastructures. On diverse multi-turn, high-concurrency agent benchmarks, SpecBox outperforms state-of-the-art serverless runtimes while preserving workflow correctness and protocol compatibility. SpecBox reduces P99 latency by up to 2.9× and peak host memory usage by 45.9%. This paper makes the following key contributions.

本文提出 SpecBox，一个对智能体工作流任务进行高效编排的预测性服务系统。SpecBox 将 LLM 执行与环境准备重叠，打破朴素智能体运行时实现中僵硬的串行依赖。SpecBox 通过三项协同技术降低端到端延迟：i）意图感知沙盒预热，基于流式 Token 输出推测执行意图，在恰当时机启动，并在同一步内将沙盒准备与正在进行的生成重叠；ii）随机沙盒预取，利用历史智能体执行轨迹预判未来沙盒需求，并在当前步骤期间为下一步准备沙盒，从而降低冷启动延迟；iii）复用感知数据传输，利用语义相似性，通过语义缓存绕过冗余工具执行，并通过带外数据路径将大型产物交付与控制信令解耦，消除不必要的计算与序列化开销。SpecBox 的原型已实现并集成到开源 AgentScope 框架（Gao et al., 2024）。SpecBox 与框架无关，可高度扩展到任意 LLM 智能体服务基础设施。在多样化的多轮、高并发智能体基准上，SpecBox 优于现有最先进的无服务器运行时，同时保持工作流正确性与协议兼容性。SpecBox 将 P99 延迟最多降低 2.9×，峰值主机内存使用降低 45.9%。本文的主要贡献如下。

  • characterizing the latency-critical path of multi-turn LLM agent execution and proposing a new sandbox prewarming approach (C1) via inferring streaming token-level intents such that sandbox preparation can be better overlapped with the ongoing LLM inference (§ 3.1).

  • 刻画多轮 LLM 智能体执行的延迟关键路径，并提出一种新的沙盒预热方法（C1）：通过推断流式 Token 级意图，使沙盒准备能更好地与正在进行的 LLM 推理重叠（§ 3.1）。

  • devising stochastic sandbox prefetching mechanism (C2) to reduce cross-step cold-start latency by mining historical agent execution traces and pre-warming likely sandbox environments over a sandbox dependency graph (§ 3.2).

  • 设计随机沙盒预取机制（C2），通过挖掘历史智能体执行轨迹，并在沙盒依赖图上预热可能的沙盒环境，降低跨步冷启动延迟（§ 3.2）。

  • introducing reuse-aware data transmission (C3), via out-of-band transport and semantic caching, efficiently transferring and reusing intermediate execution artifacts while eliminating redundant sandbox initialization (§ 3.3).

  • 引入复用感知数据传输（C3），通过带外传输与语义缓存，高效传输并复用中间执行产物，同时消除冗余沙盒初始化（§ 3.3）。

2 Background and Motivation / 背景与动机

2.1 Agent Workflow and Environment / 智能体工作流与环境

Large Language Model (LLM) applications have shifted from monolithic, single-turn chatbots to distributed autonomous agents. Traditional workflows rely on manually predefined execution graphs, where the operation sequence is fixed before runtime. In contrast, modern agent workflows integrate autonomous decision-making: given a high-level objective, an agent can dynamically decompose tasks, select execution sandboxes or external environments, and adapt its trajectory based on intermediate observations. As a result, the workflow is no longer statically specified but instead emerges from continuous interactions between LLM reasoning and external environments (Yao et al., 2022; Li et al., 2024b; Wang et al., 2024).

大语言模型（LLM）应用已从单体、单轮聊天机器人转向分布式自主智能体。传统工作流依赖人工预定义的执行图，操作序列在运行前即已固定。相反，现代智能体工作流融入自主决策：给定高层目标，智能体可动态分解任务、选择执行沙盒或外部环境，并根据中间观测调整轨迹。因此，工作流不再静态指定，而是从 LLM 推理与外部环境的持续交互中涌现（Yao et al., 2022；Li et al., 2024b；Wang et al., 2024）。

An agent workflow is realized through a sequence of event-driven execution steps, where each step represents an intermediate task that involves reasoning, planning, or invoking local and remote sandboxes. Tasks that require file system interaction, code execution, web access, or queries against proprietary databases are delegated to isolated external execution layers (environment or tool sandboxes). Standardized protocols—most notably the Model Context Protocol (MCP) (Anthropic PBC, 2024) proposed by Anthropic—formalize this interface boundary. MCP specifies a common communication layer over transports such as HTTP and Server-Sent Events (SSE), thereby transforming tool invocation into a set of loosely coupled microservices. This architecture is closely aligned with emerging agent-oriented operating systems that conceptualize LLMs as central processing units and external environments as peripheral devices (Mei et al.,; Gao et al., 2024). From a systems-engineering standpoint, an agent’s execution increasingly manifests as a continuous stream of heterogeneous, RPC-like interactions between the LLM inference core and multiple environment services, rather than as a monolithic, localized compute workload.

智能体工作流通过一系列事件驱动的执行步骤实现，每一步代表涉及推理、规划或调用本地与远程沙盒的中间任务。需要文件系统交互、代码执行、网页访问或对专有数据库查询的任务，会被委托给隔离的外部执行层（环境或工具沙盒）。标准化协议——尤其是 Anthropic 提出的模型上下文协议（MCP）（Anthropic PBC, 2024）——形式化了这一接口边界。MCP 在 HTTP 与 Server-Sent Events（SSE）等传输之上规定了公共通信层，从而将工具调用转化为一组松耦合微服务。这一架构与新兴的面向智能体的操作系统高度契合：后者将 LLM 概念化为中央处理器，将外部环境概念化为外围设备（Mei et al.,；Gao et al., 2024）。从系统工程角度看，智能体执行日益表现为 LLM 推理核心与多个环境服务之间连续的、异构的、类 RPC 交互流，而非单体的、局部化的计算工作负载。

2.2 Serving Runtimes / 服务运行时

Disaggregating Agent Engines from their underlying execution environments enables substantial scalability and architectural flexibility. However, deploying these decoupled components within cloud-native, multi-tenant cluster infrastructures introduces significant challenges related to performance and resource management. Service providers must carefully balance isolation guarantees, resource efficiency, and end-to-end latency, thereby exposing an inherent trade-off between long-running (reserved) and serverless (on-demand) serving paradigms.

将智能体引擎与其底层执行环境解耦，能够带来显著的可扩展性与架构灵活性。然而，在云原生、多租户集群基础设施中部署这些解耦组件，会在性能与资源管理方面引入重大挑战。服务提供商必须仔细平衡隔离保证、资源效率与端到端延迟，从而暴露出长运行（预留）与无服务器（按需）服务范式之间固有的权衡。

Reserved Agent Runtime. A straightforward strategy (Tan et al., 2025; Huang et al., 2026) is to maintain pre-initialized, always-on containers for each user tenant and its associated sandboxes. However, in contemporary agent ecosystems comprising thousands of fine-grained tools, sustaining all execution environments in a warmed state becomes prohibitively resource-intensive: idle containers incur substantial host memory and CPU overheads (Xu et al., 2024; Stojkovic et al., 2023a), which in turn lead to pronounced cluster underutilization and resource interference, thereby rendering large-scale multi-tenant deployments economically unsustainable.

预留式智能体运行时。 一种直接策略（Tan et al., 2025；Huang et al., 2026）是为每个用户租户及其关联沙盒维持预初始化、始终在线的容器。然而，在包含数千种细粒度工具的当代智能体生态系统中，将所有执行环境保持在预热状态会变得资源成本过高：空闲容器会带来可观的主机内存与 CPU 开销（Xu et al., 2024；Stojkovic et al., 2023a），进而导致显著的集群低利用率与资源干扰，使大规模多租户部署在经济上不可持续。

On-demand Agent Runtime. To enhance resource utilization, modern cloud-native platforms commonly employ on-demand runtime provisioning (Du et al., 2020; Shahrad et al., 2020), wherein tool sandboxes are instantiated (“cold-started”) only upon explicit requests from agents. However, this design choice introduces substantial tail latency: the on-demand initialization of an isolated container incurs a series of serialized overheads, including container image download and extraction, network namespace setup, virtual file system mounting, and application-level handshake procedures. Under bursty workloads or in complex multi-turn workflows, these multi-second cold-start delays accumulate along the agent’s end-to-end critical path, thereby inflating P99 tail latency and degrading the interactive quality of service (QoS) of real-time intelligent agents.

按需智能体运行时。 为提升资源利用率，现代云原生平台普遍采用按需运行时供给（Du et al., 2020；Shahrad et al., 2020），即仅在智能体显式请求时才实例化（“冷启动”）工具沙盒。然而，这一设计选择会引入显著的尾延迟：按需初始化隔离容器会带来一系列串行开销，包括容器镜像下载与解压、网络命名空间设置、虚拟文件系统挂载，以及应用级握手过程。在突发工作负载或复杂多轮工作流下，这些数秒级的冷启动延迟会沿智能体端到端关键路径累积，从而抬高 P99 尾延迟，并劣化实时智能体的交互式服务质量（QoS）。

2.3 Execution Bottlenecks in Agent Runtimes / 智能体运行时中的执行瓶颈

In contrast to conventional LLM serving, an agent session constitutes a continuous, stateful execution loop composed of multiple iterative reasoning and sandboxed execution steps. Prevailing agent frameworks (Gao et al., 2024; Wu et al., 2023; Langchain-ai, 2026) employ a reactive execution paradigm, in which each tool invocation is initiated only after the LLM has completed its reasoning step. This design induces a strictly serialized dependency chain between model inference and environment interaction. The resulting execution workflow is depicted in Fig. 1.

与传统 LLM 服务不同，智能体会话构成由多次迭代推理与沙盒化执行步骤组成的连续、有状态执行循环。主流智能体框架（Gao et al., 2024；Wu et al., 2023；Langchain-ai, 2026）采用反应式执行范式，即仅在 LLM 完成推理步骤后才发起每次工具调用。这一设计在模型推理与环境交互之间诱导出严格串行的依赖链。所得执行工作流如图 1 所示。

As illustrated in Fig. 2(a), the end-to-end latency of a single execution step N can be broken down into several parts:

如图 2(a) 所示，单次执行步骤 N 的端到端延迟可分解为若干部分：

The first two components are associated with LLM inference, including context processing and token generation. The remaining three define the optimization boundary for an agent runtime: Tenv_prep(N) is the time duration to get the selected sandbox environment ready, Tdata_io(N) is the time to exchange invocation inputs and results, and Tsandbox_exec(N) is the time spent executing the tool.

前两个分量与 LLM 推理相关，包括上下文处理与 Token 生成。其余三个定义了智能体运行时的优化边界：Tenv_prep(N) 是使所选沙盒环境就绪的时间，Tdata_io(N) 是交换调用输入与结果的时间，Tsandbox_exec(N) 是执行工具所花费的时间。

SpecBox targets these time frames without changing the LLM inference stack by overlapping preparation with reasoning, reducing data movement, and eliminating redundant executions. We derived the following observations that motivate this study.

SpecBox 在不改变 LLM 推理栈的前提下针对这些时间段进行优化：将准备与推理重叠、减少数据移动，并消除冗余执行。我们据此得到如下观察，构成本研究的动机。

Observation #1: In-step preparation can overlap with token generation. In today’s reactive execution model, environment initialization cannot start until the LLM emits a explicit sandbox invocation, even though the prompt, plan, and partial token stream often already reveal the likely sandbox. Using this partial evidence to start preparation earlier can put the prewarm even forward, ahead of the corresponding invocation. This interval lets sandbox startup, runtime connection setup, and other preparation work overlap with the remaining Tgeneration(N), rather than placing all of Tenv_prep(N) on the critical path. As shown in Fig. 2(b), sandbox initialization requires several seconds and can approach 20 seconds for heavyweight sandboxes, rendering this overlap mechanism critical for minimizing the effective preparation latency observed by the system.

观察 #1：步内准备可与 Token 生成重叠。 在当今的反应式执行模型中，即便提示词、计划与部分 Token 流往往已经揭示了可能的沙盒，环境初始化也必须等到 LLM 发出显式沙盒调用后才能开始。利用这些部分证据更早启动准备，可将预热前移到对应调用之前。这一间隔使沙盒启动、运行时连接建立及其他准备工作能够与剩余的 Tgeneration(N) 重叠，而非将全部 Tenv_prep(N) 放在关键路径上。如图 2(b) 所示，沙盒初始化需要数秒，重量级沙盒甚至可接近 20 秒，因此这一重叠机制对最小化系统所观测到的有效准备延迟至关重要。

Observation #2: Cross-step execution can advance the prewarming. Consecutive steps within an agent session frequently exhibit strong temporal locality. For instance, in a large language model (LLM) serving workload, a paper search step is often followed by a document reading step that processes a retrieved document, and a data analysis step is commonly followed by a figure generation step. Nonetheless, prevailing runtime systems generally treat each invocation as an independent event, discarding workflow context once a step is initiated. In contrast, the runtime can exploit the execution window of the current sandbox to proactively instantiate and prewarm the most likely subsequent execution environments before step N+1 begins, thereby making the intra-step prewarm more in-advance beyond the current decoding stage.

观察 #2：跨步执行可提前预热。 智能体会话中的连续步骤常常表现出强时间局部性。例如，在大语言模型（LLM）服务工作负载中，论文检索步骤之后往往是处理已检索文档的文档阅读步骤，而数据分析步骤之后通常是图表生成步骤。尽管如此，主流运行时系统一般将每次调用视为独立事件，一旦某步启动便丢弃工作流上下文。相反，运行时可利用当前沙盒的执行窗口，在步骤 N+1 开始之前主动实例化并预热最可能的后续执行环境，从而使步内预热超越当前解码阶段、进一步提前。

Observation #3: Redundant execution and excessive unnecessary data transfer. In multi-turn agent workflows, later reasoning steps often revisit information that was already produced, such as querying the same document again or re-running deterministic analyses. However, conventional runtimes still re-execute these operations even when an equivalent result is already available, repeatedly paying the cost of Tsandbox_exec(N). Meanwhile, sandbox interfaces often couple the synchronous transfer of large intermediate artifacts (e.g., files, images, structured outputs) with control messages, causing Tdata_io(N) to scale with artifact size. Intuitively, one can accelerate the agent execution by caching and reusing prior execution results and decoupling artifact transmission from control signaling, thereby eliminating redundant work and data transfer overhead.

观察 #3：冗余执行与过量不必要的数据传输。 在多轮智能体工作流中，后续推理步骤常常回访已经产生的信息，例如再次查询同一文档或重跑确定性分析。然而，即便等价结果已经可用，传统运行时仍会重新执行这些操作，反复支付 Tsandbox_exec(N) 的成本。与此同时，沙盒接口往往将大型中间产物（如文件、图像、结构化输出）的同步传输与控制消息耦合，使 Tdata_io(N) 随产物规模增长。直观上，可以通过缓存并复用先前执行结果，并将产物传输与控制信令解耦，从而加速智能体执行，消除冗余工作与数据传输开销。

These observations reveal optimization opportunities at three points in the agent execution loop: overlapping the environment preparation with the LLM operations within a step, advancing the environment preparation of the next step in the current step, and eliminating repeated execution or data movement. Existing LLM serving frameworks primarily focus on optimizing model execution — such as prefill and decoding phases, as well as KV cache management — while serverless runtimes primarily reduce the overhead associated with container initialization without considering user-wise intent. However, neither of them directly addresses how an agent-oriented runtime can leverage these optimizations while maintaining resource efficiency and preserving the intended semantics of tool invocation and usage.

这些观察揭示了智能体执行循环中三个优化切入点：在同一步内将环境准备与 LLM 操作重叠；在当前步骤中提前下一步的环境准备；以及消除重复执行或数据移动。现有 LLM 服务框架主要聚焦于优化模型执行——如预填充与解码阶段，以及 KV 缓存管理——而无服务器运行时主要降低容器初始化开销，却不考虑用户侧意图。然而，两者都没有直接回答：面向智能体的运行时如何在利用这些优化的同时，保持资源效率并保留工具调用与使用的既定语义。

EN Figure 2: Execution bottlenecks in agent runtimes. (a) Execution-step latency breakdown. (b) Mean cold-start latency across 32 sandboxed environments, including all MCPBench (Wang et al., 2025) sandboxes and additional commonly used environments. Most sandboxes initialize in 2–4 seconds, while resource-intensive environments can require up to approximately 20 seconds. CN 图 2: 智能体运行时中的执行瓶颈。(a) 执行步骤延迟分解。(b) 32 个沙盒化环境的平均冷启动延迟，包括全部 MCPBench（Wang et al., 2025）沙盒及其他常用环境。大多数沙盒在 2–4 秒内完成初始化，而资源密集型环境可能需要大约 20 秒。

EN Figure 3: Intent-aware sandbox prewarming: the plan agent expands user requests into ReAct steps, and keyword plus semantic predictions jointly decide to prewarm the sandbox or not. CN 图 3: 意图感知沙盒预热：规划智能体将用户请求展开为 ReAct 步骤，关键词预测与语义预测共同决定是否预热沙盒。

EN Figure 4: Tradeoffs among routing policies. Keyword routing trades early decisions for potential false positives; semantic routing and intersection delay a precise decision; union assembly retains precision while allowing either router to trigger prewarming early. CN 图 4: 路由策略之间的权衡。关键词路由以潜在假阳性换取早期决策；语义路由与交集会推迟精确决策；并集组装在保留精度的同时，允许任一路由器尽早触发预热。

2.4 Research Challenges / 研究挑战

Designing an effective execution runtime to accelerate the LLM agent serving is faced with the following challenges.

设计有效的执行运行时以加速 LLM 智能体服务，面临如下挑战。

Challenge #1: Early intent prediction vs. resource waste. Intent speculation of tool use from given incomplete token stream is critical to ensure the timeliness of sandbox environment preparation whilst avoiding unwanted tool launch due to inaccurate intent prediction. Early warmup must rely on highly ambiguous early-stage tokens (e.g., general-purpose verbs such as read and get reused across different MCP tools), which often causes false-positive sandbox activation, wasting resources and potentially triggering host-level OOM in multi-tenant deployments.

挑战 #1：早期意图预测 vs. 资源浪费。 从给定的不完整 Token 流中推测工具使用意图，对于保证沙盒环境准备的及时性、同时避免因意图预测不准而启动不需要的工具至关重要。早期预热必须依赖高度模糊的早期 Token（例如在不同 MCP 工具间复用的通用动词，如 read 与 get），这常常导致假阳性沙盒激活，浪费资源，并可能在多租户部署中触发主机级 OOM。

Challenge #2: Cross-step in-advance prewarming vs. non-deterministic workflow execution. It is advantageous to select the subsequent-step sandbox environments in advance of the manifestation of the next intent. The agent workflow executes in a probabilistic manner, admitting multiple plausible successor states. Prewarming all potential successors incurs a computational and resource overhead comparable to that of a fully reserved deployment, whereas prewarming only a single successor substantially reduces the opportunity to amortize sandbox initialization costs and yields only limited latency improvements. Instead, the runtime system should exploit historical transition data to rank a small set of highly probable target environments, then refine this choice using the signal available at the current step. This balances resource utilization against latency reduction while preserving the ability to conceal sandbox initialization behind ongoing computation.

挑战 #2：跨步提前预热 vs. 非确定性工作流执行。 在下一意图显现之前选定后续步骤的沙盒环境是有利的。智能体工作流以概率方式执行，允许多种合理的后继状态。对所有潜在后继都预热，会带来与完全预留部署相当的计算与资源开销；而只预热单一后继，则会大幅降低摊销沙盒初始化成本的机会，仅带来有限的延迟改善。相反，运行时系统应利用历史转移数据对一小集合高度可能的目标环境排序，再用当前步骤可用的信号精炼这一选择。这在资源利用率与延迟降低之间取得平衡，同时保留将沙盒初始化隐藏在正在进行的计算之后的能力。

Challenge #3: Reuse and data-path efficiency vs. sandbox compatibility. We must skip redundant work without changing what the agent observes from a sandbox invocation. Exact reuse is safe but misses semantically equivalent requests expressed differently; unconstrained semantic reuse may yield incompatible results. Similarly, putting large artifacts in normal RPC messages preserves compatibility but keeps data movement proportional to payload size, while replacing the control interface would break existing tools. Hence, the runtime must detect compatible reuse, reject unsafe approximations, and decouple bulky artifacts from control signaling while preserving sandbox-execution semantics.

挑战 #3：复用与数据路径效率 vs. 沙盒兼容性。 我们必须跳过冗余工作，同时不改变智能体从沙盒调用中观察到的内容。精确复用是安全的，但会错过以不同表述表达的语义等价请求；不受约束的语义复用可能产生不兼容的结果。类似地，将大型产物放入普通 RPC 消息可保持兼容性，但数据移动仍与载荷大小成正比；而替换控制接口则会破坏现有工具。因此，运行时必须检测兼容复用、拒绝不安全近似，并将大宗产物与控制信令解耦，同时保留沙盒执行语义。

3 Design of SpecBox / SpecBox 的设计

We focus on three critical time periods within the execution of reactive agents: environment preparation, data movement, and sandbox execution. This section how SpecBox accelerate them with decoupled yet inter-connected optimizations: intent-aware sandbox prewarming that infers an in-step tool intent early to overlap the environment preparation with the decoding of LLM serving (§ 3.1); stochastic sandbox prefetching that exploits cross-step workflow regularity to further put the preparation forward (§ 3.2); and reuse-aware data transmission that avoids redundant execution and removes large artifacts from the control path (§ 3.3).

我们关注反应式智能体执行中的三个关键时间段：环境准备、数据移动与沙盒执行。本节说明 SpecBox 如何通过解耦而又相互关联的优化来加速它们：意图感知沙盒预热，尽早推断步内工具意图，使环境准备与 LLM 服务的解码重叠（§ 3.1）；随机沙盒预取，利用跨步工作流规律性进一步将准备前移（§ 3.2）；以及复用感知数据传输，避免冗余执行并将大型产物移出控制路径（§ 3.3）。

EN Figure 5: Stochastic sandbox prefetching in SpecBox. Execution history predicts likely sandboxes for subsequent ReAct steps. CN 图 5: SpecBox 中的随机沙盒预取。执行历史预测后续 ReAct 步骤可能需要的沙盒。

3.1 Intent-Aware Sandbox Prewarming / 意图感知沙盒预热

Naive reactive runtimes await a full tool invocation before sandbox preparation, even if prompts, evolving plan, and partial generation already reveal user intents. SpecBox instead prewarms the sandbox earlier by inferring streaming token-level intents, better overlapping sandbox preparation with ongoing LLM inference.

朴素的反应式运行时在沙盒准备之前等待完整的工具调用，即便提示词、演化中的计划与部分生成已经揭示了用户意图。SpecBox 则通过推断流式 Token 级意图更早地预热沙盒，使沙盒准备更好地与正在进行的 LLM 推理重叠。

SpecBox makes the best use of the LLM docoding period Tgeneration(N) to obtain just enough user intents for launching accurate sandboxes without compromising the timeliness of prewarming. This is done by streaming context to two independent routers. As shown in Fig. 4, there exists a dilemma: earlier intent predictions overlap more with the LLM decoding period but are less reliable and can waste sandbox capacity. Keyword Router acts early but must balance false positives against a stricter threshold; Semantic Router is more precise but slower. Requiring both predictions incurs the Semantic Router’s delay, whereas opportunistically combining their outputs preserves high precision while allowing earlier decisions. Hence, SpecBox proposes an intent-aware sandbox prewarming mechanism that combines the two asynchronous intent predictions, rather than considering either one on its own to be sufficient.

SpecBox 充分利用 LLM 解码时段 Tgeneration(N)，获取刚好足够的用户意图以准确启动沙盒，同时不损害预热的及时性。做法是将流式上下文送入两个独立路由器。如图 4 所示，存在一个两难：更早的意图预测与 LLM 解码时段重叠更多，但更不可靠，并可能浪费沙盒容量。关键词路由器行动较早，但必须在假阳性与更严格阈值之间权衡；语义路由器更精确但更慢。要求两者同时预测会引入语义路由器的延迟，而机会主义地组合其输出则可在允许更早决策的同时保持高精度。因此，SpecBox 提出一种意图感知沙盒预热机制，组合这两种异步意图预测，而非认为单独任一者即已足够。

Keyword Router. The Keyword Router scans the stream against tool-specific keyword profiles and can emit a candidate within microseconds of a distinctive token, preparing the environment while the LLM continues decoding. Common terms like research, search, or slide occur in many tool descriptions: triggering on a single match creates a large prewarm set with many false positives, while requiring many matches will delay the preparation, missing good chances of overlapping. We therefore apply a threshold γ on the number of matched tool-specific keywords. In Fig. 3, surface cues in the user request and evolving plan let the Keyword Router emit likely sandbox candidates before the plan agent finishes the current ReAct step. The threshold balances early activation with resource waste; the chosen configuration is given in § 5.3.

关键词路由器。 关键词路由器对照工具特定的关键词画像扫描数据流，并可在出现辨识性 Token 后的微秒级发出候选，使环境在 LLM 继续解码时完成准备。research、search 或 slide 等常见术语出现在许多工具描述中：仅凭单次匹配触发会产生带有大量假阳性的庞大预热集合，而要求多次匹配则会推迟准备，错过良好的重叠机会。因此，我们对匹配到的工具特定关键词数量施加阈值 γ。在图 3 中，用户请求与演化计划中的表层线索使关键词路由器能够在规划智能体完成本次 ReAct 步骤之前发出可能的沙盒候选。该阈值在早期激活与资源浪费之间取得平衡；所选配置见 § 5.3。

Semantic Router. In parallel, the Semantic Router compares the active context with tool-intent representations. It captures requests whose wording overlaps little with a tool profile and disambiguates generic keyword cues using the plan and prior generation. In Fig. 3, it can identify candidate sandboxes from the broader task intent even when no single token uniquely identifies a tool. Its tradeoff is temporal: reliable semantic evidence typically needs a longer prefix, so a semantic-only decision often arrives too late to hide the cold start latency of a heavy-weight sandbox such as PaperSearch (OpenAGS, 2026), and Neo4j (Neo4j, 2026). The Semantic Router is an asynchronous, complementary source of candidates that can recover intents the Keyword Router misses.

语义路由器。 并行地，语义路由器将活动上下文与工具意图表示进行比较。它能捕捉措辞与工具画像重叠很少的请求，并利用计划与先前生成来消歧通用关键词线索。在图 3 中，即便没有单个 Token 能唯一标识某工具，它也能从更广泛的任务意图中识别候选沙盒。其权衡是时间性的：可靠的语义证据通常需要更长的前缀，因此仅依赖语义的决策往往来得太晚，无法掩盖 PaperSearch（OpenAGS, 2026）与 Neo4j（Neo4j, 2026）等重量级沙盒的冷启动延迟。语义路由器是候选的异步、互补来源，可恢复关键词路由器错过的意图。

Union Assembly. Let 𝒮key(N) and 𝒮semantic(N) be the candidate sets from the two routers at token step N. The lower-left quadrant of Fig. 4 shows why prewarming only their intersection is unsuitable: it boosts apparent precision but makes every trigger wait for the Semantic Router and drops valid tools whenever either router has imperfect recall. Instead, we use the lower-right policy:

并集组装。 令 𝒮key(N) 与 𝒮semantic(N) 为两个路由器在 Token 步骤 N 的候选集。图 4 左下象限说明了为何仅预热其交集并不合适：它提升了表面精度，却使每次触发都等待语义路由器，并且只要任一路由器召回不完善就会丢弃有效工具。相反，我们使用右下策略：

Each router manages its own false positives, and the union lets the first credible signal start preparation. Fig. 3 shows the resulting behavior: routers independently generate candidates from the request and partial plan, then form a unified prewarm set for the sandbox manager. Explicit intents benefit from the early prewarm of the Keyword Router, while implicit intents are handled by the Semantic Router. As will be shown in § 5.3, asynchronous union can minimize waiting time and eliminate cold starts in the evaluated routing workload.

每个路由器各自管理自身的假阳性，并集则让第一个可信信号即可启动准备。图 3 展示了所得行为：路由器从请求与部分计划独立生成候选，再形成统一的预热集合交给沙盒管理器。显式意图受益于关键词路由器的早期预热，而隐式意图由语义路由器处理。如 § 5.3 将展示的，异步并集可最小化等待时间，并在所评估的路由工作负载中消除冷启动。

EN Figure 6: Stochastic sandbox prefetching using a first-order Markov model. Left: SDG-based Markov state transition graph with edge probabilities from observed counts. Right: example session showing thresholded, budgeted prefetching across turns, where predicted successors are prewarmed before the next step commits. CN 图 6: 使用一阶马尔可夫模型的随机沙盒预取。左：基于 SDG 的马尔可夫状态转移图，边概率来自观测计数。右：示例会话，展示跨轮次的阈值化、有预算预取，其中预测后继在下一步提交之前被预热。

3.2 Stochastic Sandbox Prefetching / 随机沙盒预取

Intent-aware sandbox prewarming can only use the remaining decoding time of the current step, which is inadequate when a non-resident sandbox takes longer to start than the marginal token-generation window. In reality, steps in an agent workflow are non-deterministic but generally follow a probabilistic model. For instance, after a paper search, an agent may read a returned document; after data analysis, it may generate a figure or report. SpecBox exploits these cross-step probabilistic patterns to navigate the environment preparation in advance during Tsandbox_exec(N), before the next step commits to a specific invocation.

意图感知沙盒预热只能利用当前步骤剩余的解码时间；当非驻留沙盒的启动时间长于边际 Token 生成窗口时，这并不充分。实际上，智能体工作流中的步骤虽非确定性，但通常遵循概率模型。例如，论文检索之后，智能体可能阅读返回的文档；数据分析之后，它可能生成图表或报告。SpecBox 利用这些跨步概率模式，在 Tsandbox_exec(N) 期间、下一步提交到具体调用之前，提前导航环境准备。

Stochastic Markov Process Modeling. Let 𝒱 denote the sandbox state space, where each state is a sandbox type or a typed tool-state tuple in the state-level SDG. For an execution trace {S(1),…,S(N)}, SpecBox records step-to-step transitions in a directed sandbox dependency graph (SDG). For each ordered pair (vi,vj), we maintain transition counts:

随机马尔可夫过程建模。 令 𝒱 表示沙盒状态空间，其中每个状态是沙盒类型，或状态级 SDG 中的带类型工具—状态元组。对于执行轨迹 {S(1),…,S(N)}，SpecBox 在有向沙盒依赖图（SDG）中记录步到步的转移。对每个有序对 (vi,vj)，我们维护转移计数：

The next-state probability is estimated by a first-order Markov model with Laplace Smoothing (Laplace, 1812):

下一状态概率由带拉普拉斯平滑（Laplace, 1812）的一阶马尔可夫模型估计：

where α=1 avoids zero-probability collapse for sparsely observed transitions.

其中 α=1，避免稀疏观测转移出现零概率坍缩。

Prefetching Under Budget Constraints. Given current state vi, SpecBox first identifies sandbox candidates with non-trivial cold-start cost:

预算约束下的预取。 给定当前状态 vi，SpecBox 首先识别具有不可忽略冷启动成本的沙盒候选：

where Lj denotes the estimated cold-start penalty of sandbox vj and λ is a lightweight cost threshold.

其中 Lj 表示沙盒 vj 的估计冷启动惩罚，λ 是轻量成本阈值。

SpecBox then filters out low-confidence successors and retains only high-probability candidates:

随后 SpecBox 过滤低置信后继，仅保留高概率候选：

where τ controls false-positive prewarming. Finally, SpecBox selects the top-B sandboxes under a fixed budget:

其中 τ 控制假阳性预热。最后，SpecBox 在固定预算下选择 top-B 沙盒：

where 𝒞i is the filtered candidate set, 𝒜i is the final budgeted prefetch set, and B bounds the number of sandboxes prewarmed per step. This policy is intentionally lightweight and can be executed outside the LLM generation critical path.

其中 𝒞i 是过滤后的候选集，𝒜i 是最终有预算的预取集，B 限制每步预热的沙盒数量。该策略有意保持轻量，可在 LLM 生成关键路径之外执行。

Online Update. After each completed sandbox invocation, SpecBox appends one transition edge to SDG and updates Ci,j and Pi,j asynchronously in the background prefetch worker. Thus, the predictor continuously adapts to evolving multi-turn workflows without blocking foreground agent execution. Fig. 5 shows how stochastic sandbox prefetching operates together with intent-aware sandbox prewarming in a complete multi-step workflow. After step N finishes in sandbox vi, SpecBox queries the SDG for likely successor sandboxes, filters out low-value or low-confidence candidates using Lj≥λ and Pi,j≥τ, and then selects a budgeted prefetch set 𝒜i before step N+1 is fully committed.

在线更新。 每次沙盒调用完成后，SpecBox 向 SDG 追加一条转移边，并在后台预取工作线程中异步更新 Ci,j 与 Pi,j。因此，预测器可持续适应演化中的多轮工作流，而不阻塞前台智能体执行。图 5 展示了随机沙盒预取如何与意图感知沙盒预热在完整多步工作流中协同运作。步骤 N 在沙盒 vi 中结束后，SpecBox 查询 SDG 以获得可能的后继沙盒，使用 Lj≥λ 与 Pi,j≥τ 过滤低价值或低置信候选，然后在步骤 N+1 完全提交之前选择有预算的预取集 𝒜i。

EN Figure 7: Reuse-aware data transmission in SpecBox. A semantic-cache hit returns a prior result (paper_search); a miss executes the sandbox (paper_slides) and delivers the result through the out-of-band data path. CN 图 7: SpecBox 中的复用感知数据传输。语义缓存命中则返回先前结果（paper_search）；未命中则执行沙盒（paper_slides），并通过带外数据路径交付结果。

3.3 Reuse-Aware Data Transmission / 复用感知数据传输

Once an invocation is ready, its remaining critical-path cost lies in artifact movement (Tdata_io(N)) and tool execution (Tsandbox_exec(N)). SpecBox first checks if it can reuse a cached result; otherwise, it runs the tool and returns its artifact via a separate data path. Fig. 7 details this procedure.

一旦调用就绪，其剩余关键路径成本便落在产物移动（Tdata_io(N)）与工具执行（Tsandbox_exec(N)）上。SpecBox 首先检查能否复用缓存结果；否则运行工具，并通过独立数据路径返回其产物。图 7 详述了该过程。

Semantic Caching. Multi-turn agents often re-access unchanged documents, submit semantically equivalent queries with different wording, or re-run deterministic computations. Exact argument matching is robust but fails to capture such superficial variations, while unconstrained semantic matching can mistakenly merge requests that differ in tools, inputs, or side effects. Thus, caching must broaden the set of reusable requests while strictly preserving functional equivalence and behavioral compatibility.

语义缓存。 多轮智能体常常重新访问未变化的文档、以不同措辞提交语义等价查询，或重跑确定性计算。精确参数匹配稳健，但无法捕捉这类表层变化；而不受约束的语义匹配可能错误合并在工具、输入或副作用上不同的请求。因此，缓存必须扩大可复用请求集合，同时严格保持功能等价与行为兼容。

For each completed deterministic invocation, SpecBox stores a normalized invocation signature and its result. During lookup, it first filters cache entries by tool identity, then compares normalized invocation representations are compared against cached signatures. For a request x, result reuse is allowed only if a compatible cache entry exists that satisfies the semantic equivalence constraint:

对每次完成的确定性调用，SpecBox 存储规范化调用签名及其结果。查找时，它首先按工具身份过滤缓存条目，再将规范化调用表示与缓存签名比较。对于请求 x，仅当存在满足语义等价约束的兼容缓存条目时，才允许结果复用：

where x denotes the incoming tool invocation request and xi denotes the i-th cached invocation. Function ϕ⁡(⋅) transforms an invocation into a normalized semantic representation by first removing superficial variations in argument representation and then extracting semantic features for similarity comparison. Function sim(⋅,⋅) measures the similarity between two invocation representations, and the threshold τc controls the strictness of semantic reuse. The tool identity constraint guarantees interface-level compatibility, while the similarity threshold limits reuse to invocations with sufficiently similar normalized semantics.

其中 x 表示传入的工具调用请求，xi 表示第 i 条缓存调用。函数 ϕ(·) 将调用变换为规范化语义表示：先去除参数表示中的表层变化，再提取语义特征用于相似性比较。函数 sim(·,·) 度量两个调用表示之间的相似性，阈值 τc 控制语义复用的严格程度。工具身份约束保证接口级兼容，而相似性阈值将复用限制在规范化语义足够相似的调用上。

This design treats semantic matching as a conservative extension of exact reuse rather than an unconstrained approximation mechanism. Semantic similarity only enlarges the reusable request space within the boundary of the same tool interface and deterministic invocation behavior. On a cache miss, execution proceeds along the standard path and appends the fully computed result to the cache. On a cache hit, both sandbox initialization and tool invocation are skipped, thereby reducing Tenv_prep(N) and Tsandbox_exec(N). As will be demonstrated in § 5.3.3, the semantic caching strategy can recover a larger proportion of redundant computation than using the strategy of exact matching alone, while maintaining a conservative fallback path that preserves the inference correctness.

这一设计将语义匹配视为精确复用的保守扩展，而非不受约束的近似机制。语义相似性仅在同一工具接口与确定性调用行为的边界内扩大可复用请求空间。缓存未命中时，执行沿标准路径进行，并将完整计算结果追加到缓存。缓存命中时，沙盒初始化与工具调用都被跳过，从而降低 Tenv_prep(N) 与 Tsandbox_exec(N)。如 § 5.3.3 将展示的，语义缓存策略可比单独使用精确匹配恢复更大比例的冗余计算，同时维持保守回退路径以保持推理正确性。

Out-of-Band Data Transmission. Even after a sandbox is fully initialized, conventional RPC transports serialize large logs, files, images, and structured outputs into request–response messages. Consequently, Tdata_io(N) scales with the payload size and blocks the agent prior to its subsequent reasoning step. A wholesale replacement of the control protocol would compromise compatibility with existing MCP tools; therefore, SpecBox instead decouples control signaling from artifact transfer.

带外数据传输。 即便沙盒已完全初始化，传统 RPC 传输仍会将大型日志、文件、图像与结构化输出序列化进请求—响应消息。因此，Tdata_io(N) 随载荷大小增长，并在后续推理步骤之前阻塞智能体。整体替换控制协议会损害与现有 MCP 工具的兼容性；因此，SpecBox 转而将控制信令与产物传输解耦。

The in-band control plane continues to carry standard tool directives, completion notifications, error reports, and compact metadata. Large artifacts are represented on this plane solely by a fixed-size reference and are exchanged via a co-located, zero-copy data path. Under this design, a cache miss executes as usual, publishes its artifact exactly once, and returns a reference to the Agent Engine; a cache hit simply reuses and returns the previously published artifact over the same data path. This architectural decoupling maintains the existing control interface while eliminating RPC-layer serialization and memory copying for high-volume outputs. As demonstrated in § 5.3.4, the corresponding transmission latency becomes effectively insensitive to payload size.

带内控制面继续承载标准工具指令、完成通知、错误报告与紧凑元数据。大型产物在该平面上仅以固定大小的引用表示，并通过共置的零拷贝数据路径交换。在此设计下，缓存未命中按常执行、恰好发布一次产物，并向智能体引擎返回引用；缓存命中则简单地复用并经同一数据路径返回先前已发布的产物。这一架构解耦在保持现有控制接口的同时，消除了高容量输出在 RPC 层的序列化与内存拷贝。如 § 5.3.4 所示，相应传输延迟实际上对载荷大小不敏感。

4 Implementation / 实现

EN Figure 8: Overview of SpecBox. CN 图 8: SpecBox 概览。

SpecBox is implemented as a predictive serving runtime between the agent engine and external execution sandboxes. Figure 8 shows its control and data paths. The implementation observes an agent run-loop without changing its tool-facing interface, and overlaps runtime work with the LLM and sandbox work already in progress.

SpecBox 实现为位于智能体引擎与外部执行沙盒之间的预测性服务运行时。图 8 展示了其控制路径与数据路径。该实现对智能体运行循环进行观测，而不改变其面向工具的接口，并将运行时工作与已经在进行的 LLM 及沙盒工作重叠。

Control Plane. On the control plane, a controller subscribes to the incoming token stream and records completed tool transitions. The Keyword Router matches the stream against tool keyword profiles and emits a candidate when the number of matched tool-specific keywords reaches γ=2, the operating point selected in Section 5.3. In parallel, the Semantic Router uses the sparse retrieval configuration selected in that section. Union Assembly dispatches their candidate sets independently to the sandbox manager, which deduplicates requests and starts the corresponding containers. A background prefetch worker updates the sandbox dependency graph from execution traces, applies the transition probabilities in Section 3.2, and submits budgeted next-step warmups while the current tool is executing. In our implementation, we configure the cost threshold as λ=5. prefetch probability threshold as τ=0.6 and the per-step prefetch budget as B=1.

控制面。 在控制面上，控制器订阅传入的 Token 流并记录已完成的工具转移。关键词路由器对照工具关键词画像匹配数据流，当匹配到的工具特定关键词数量达到 γ=2（第 5.3 节选定的工作点）时发出候选。并行地，语义路由器使用该节选定的稀疏检索配置。并集组装将其候选集独立分派给沙盒管理器，后者对请求去重并启动相应容器。后台预取工作线程根据执行轨迹更新沙盒依赖图，应用第 3.2 节的转移概率，并在当前工具执行期间提交有预算的下一步预热。在我们的实现中，成本阈值配置为 λ=5，预取概率阈值为 τ=0.6，每步预取预算为 B=1。

Data Path. Before scheduling a deterministic invocation, the runtime constructs its invocation representation, restricts lookup to the same tool identity, and checks its semantic-result cache. Each cache entry contains the tool identifier, normalized invocation signature, semantic embedding, and result reference. The cache reuses a previous result only when the tool identity matches and the semantic similarity exceeds the threshold τc=0.8 used in the reported experiments. On a miss, the sandbox writes a large result into a host-managed memory-mapped shared-memory region. The control plane carries only a 64-bit token_id that names this region, while the Agent Engine reads the result directly from the shared-memory backplane. Small directives, metadata, completion notifications, and errors remain on the normal control plane. This implements the common cache-or-execute flow in Figure 7 without serializing bulky results through the RPC stack.

数据路径。 在调度确定性调用之前，运行时构造其调用表示，将查找限制在同一工具身份，并检查语义结果缓存。每条缓存条目包含工具标识符、规范化调用签名、语义嵌入与结果引用。仅当工具身份匹配且语义相似性超过报告实验中使用的阈值 τc=0.8 时，缓存才复用先前结果。未命中时，沙盒将大型结果写入主机管理的内存映射共享内存区域。控制面仅携带命名该区域的 64 位 token_id，而智能体引擎直接从共享内存底板读取结果。小型指令、元数据、完成通知与错误仍走普通控制面。这实现了图 7 中常见的“缓存或执行”流程，而无需通过 RPC 栈序列化大宗结果。

Execution boundary and correctness. We consider multi-tenant agents whose tools run in OS-isolated containers or microVMs. Predictive preparation makes an environment ready but does not execute side-effecting work before the agent commits the invocation; unused warmups are discarded. Cache reuse is limited to deterministic, compatible tool requests. The shared-memory bridge operates inside one trusted host or securely managed cluster boundary, with existing access controls preventing cross-tenant memory access. Sandbox escapes, malicious agents, and compromised infrastructure are outside this work’s threat model.

执行边界与正确性。 我们考虑多租户智能体，其工具运行在操作系统隔离的容器或微虚拟机中。预测性准备使环境就绪，但在智能体提交调用之前并不执行具有副作用的工作；未使用的预热会被丢弃。缓存复用限于确定性、兼容的工具请求。共享内存桥接运行在单一受信任主机或安全托管的集群边界内，现有访问控制防止跨租户内存访问。沙盒逃逸、恶意智能体与被攻破的基础设施不在本工作的威胁模型之内。

5 Evaluation / 评估

5.1 Experiment Setup / 实验设置

Hardware and Software Environment. All experiments are conducted on a commodity server equipped with an 16-core CPU, 256 GiB of host memory, and a 2 TB NVMe SSD. SpecBox is built upon the AgentScope (Gao et al., 2024) framework, specifically integrated with its agent engine layer. The isolated multi-tenant execution sandboxes are instantiated via Docker containers, and the entire runtime infrastructure is implemented in Python. To drive agent reasoning, we utilize Alibaba DashScope’s Qwen3.5-Max model (Alibaba Cloud, 2026), accessed concurrently via its production cloud API endpoints.

硬件与软件环境。 全部实验在一台商用服务器上进行，配备 16 核 CPU、256 GiB 主机内存与 2 TB NVMe SSD。SpecBox 构建于 AgentScope（Gao et al., 2024）框架之上，具体集成于其智能体引擎层。隔离的多租户执行沙盒通过 Docker 容器实例化，整个运行时基础设施以 Python 实现。为驱动智能体推理，我们使用阿里云百炼的 Qwen3.5-Max 模型（Alibaba Cloud, 2026），通过其生产云 API 端点并发访问。

Workloads. We evaluate SpecBox using a trace-level benchmark with 200 multi-turn trajectories, derived from MCPBench (Wang et al., 2025) with 32 open-source MCP-compatible tool servers (e.g., Playwright (Microsoft, 2026b), Jupyter (Datalayer, 2026), Neo4j (Neo4j, 2026)) collected from GitHub (GitHub, 2026). To ensure tool-level validity and realistic workflow execution dependencies, we construct the benchmark in two stages:

工作负载。 我们使用轨迹级基准评估 SpecBox，包含 200 条多轮轨迹，源自 MCPBench（Wang et al., 2025），涵盖从 GitHub（GitHub, 2026）收集的 32 个开源 MCP 兼容工具服务器（例如 Playwright（Microsoft, 2026b）、Jupyter（Datalayer, 2026）、Neo4j（Neo4j, 2026））。为确保工具级有效性与现实的工作流执行依赖，我们分两阶段构建该基准：

  • Atomic Tool-use Generation: We generate 20 single-turn interaction templates per tool (640 in total) to cover diverse atomic behaviors across all 32 tools.

  • 原子工具使用生成： 我们为每个工具生成 20 个单轮交互模板（共 640 个），以覆盖全部 32 个工具的多样化原子行为。

  • Multi-turn Trajectory Construction: An LLM planner incrementally generates 1–10 step agent sessions, where each planned task is executed against actual servers to get real execution results back into subsequent planning steps.

  • 多轮轨迹构建： LLM 规划器增量生成 1–10 步的智能体会话，其中每个规划任务都针对真实服务器执行，并将真实执行结果回馈到后续规划步骤。

The resulting dataset is uniformly distributed with 20 traces per trajectory length, averaging 6.4 steps per session and 2.96 tools per step. This execution-grounded approach ensures tool-level validity and eliminates tool-specific bias while reflecting representative LLM agent workflows.

所得数据集按轨迹长度均匀分布，每种长度 20 条轨迹，平均每会话 6.4 步、每步 2.96 个工具。这种以执行为根基的方法确保了工具级有效性，消除了工具特定偏差，同时反映了有代表性的 LLM 智能体工作流。

Methodology. Unless otherwise specified, we evaluate all systems by replaying full traces as session-level workloads with a fixed random seed (seed = 0). Each trace is executed in a step-wise manner, preserving dependencies across reasoning, tool invocation, and sandbox execution to faithfully model realistic multi-turn agent workflows. Under this deterministic sampling setting, the resulting trace distribution naturally exhibits a concentration in the 5–8 step range, which we therefore treat as the representative workload regime rather than a manually selected subset.

方法。 除非另有说明，我们以固定随机种子（seed = 0）将完整轨迹作为会话级工作负载回放，以评估所有系统。每条轨迹逐步执行，保留推理、工具调用与沙盒执行之间的依赖，以忠实建模现实的多轮智能体工作流。在这一确定性采样设置下，所得轨迹分布自然集中在 5–8 步区间，因此我们将其视为代表性工作负载区间，而非人工选定的子集。

We evaluate all experiments at the session level unless explicitly stated otherwise, and defer workload-specific configurations (e.g., trace length or sampling range) to each individual ablation study.

除非明确另行说明，我们在会话级评估全部实验，并将工作负载特定配置（如轨迹长度或采样范围）留待各消融研究自行说明。

EN Figure 9: Cumulative sandbox provisioning latency across multi-turn agent sessions (5–8 steps per session). CN 图 9: 多轮智能体会话（每会话 5–8 步）的累积沙盒供给延迟。

EN Figure 10: End-to-end performance and resource consumption under concurrent workloads: (a) P99 E2E latency, (b) mean cumulative sandbox provisioning latency, (c) peak CPU usage, and (d) peak memory usage. CN 图 10: 并发工作负载下的端到端性能与资源消耗：(a) P99 端到端延迟，(b) 平均累积沙盒供给延迟，(c) 峰值 CPU 使用，(d) 峰值内存使用。

Metrics. We evaluate SpecBox across three dimensions aligned with system design goals:

指标。 我们从与系统设计目标对齐的三个维度评估 SpecBox：

  • End-to-end Latency: Per agent session, we report mean and tail latency (P99) to capture both average performance and long-tail behavior under multi-step execution.

  • 端到端延迟： 对每个智能体会话，我们报告平均延迟与尾延迟（P99），以捕捉多步执行下的平均性能与长尾行为。

  • Resource Efficiency: We measure peak CPU and memory consumption under multi-tenant workloads to quantify the runtime overhead introduced by sandbox provisioning, execution, and lifecycle management.

  • 资源效率： 我们在多租户工作负载下测量峰值 CPU 与内存消耗，以量化沙盒供给、执行与生命周期管理引入的运行时开销。

  • Prediction Accuracy: We evaluate the accuracy of all predictive runtime optimizations, including intent-aware sandbox prewarming, stochastic sandbox prefetching, and semantic cache. For each, we report i) correct triggering or retrieval decisions under uncertain tool intent, and ii) false positive rate, reflecting unnecessary or incorrect activations such as misrouted intents, redundant prewarming.

  • 预测精度： 我们评估全部预测性运行时优化的精度，包括意图感知沙盒预热、随机沙盒预取与语义缓存。对每一项，我们报告 i）在不确定工具意图下正确的触发或检索决策，以及 ii）假阳性率，反映不必要或不正确的激活，如误路由意图、冗余预热。

5.2 End-to-End Performance / 端到端性能

This section evaluates the end-to-end (E2E) performance of SpecBox under two representative baselines in agent serving. We define our baselines as follows:

本节在智能体服务的两个代表性基线下评估 SpecBox 的端到端（E2E）性能。基线定义如下：

  • Reserved Runtime: Maintains permanently warm sandboxes for all candidate tools. This represents the latency lower bound (performance ceiling). However, it is financially and physically non-viable in production due to prohibitive idle memory footprint across massive, multi-tenant MCP tool ecosystems.

  • 预留运行时： 为所有候选工具维持永久预热沙盒。这代表延迟下界（性能上限）。然而，由于在大规模、多租户 MCP 工具生态系统中空闲内存占用过高，它在生产中财务上与物理上均不可行。

  • On-demand Runtime: Instantiates sandboxes dynamically upon tool calls, mirroring production MCP runtimes (Amazon Web Services, 2026; Microsoft, 2026a; Google Cloud, 2026). Since its bottleneck stems from application-level tool binding and session handshakes rather than generic OS booting, it remains the standard practical baseline.

  • 按需运行时： 在工具调用时动态实例化沙盒，镜像生产 MCP 运行时（Amazon Web Services, 2026；Microsoft, 2026a；Google Cloud, 2026）。由于其瓶颈源于应用级工具绑定与会话握手，而非通用操作系统启动，它仍是标准的实用基线。

We aim to answer a fundamental question: Can SpecBox successfully decouple execution latency from physical resource constraints, achieving serverful-like performance with serverless-like cost?

我们旨在回答一个根本问题：SpecBox 能否成功将执行延迟与物理资源约束解耦，以接近有服务器的性能、接近无服务器的成本？

5.2.1 Cumulative Sandbox Provisioning Latency in Multi-Turn Agent Workflows / 多轮智能体工作流中的累积沙盒供给延迟

We evaluate the end-to-end impact of SpecBox on sandbox provisioning latency over multi-turn agent execution traces. We define cumulative sandbox provisioning latency as the sum of sandbox initialization delays incurred at each tool invocation within a session, focusing exclusively on environment setup overheads and excluding in-sandbox computation time.

我们评估 SpecBox 对多轮智能体执行轨迹上沙盒供给延迟的端到端影响。我们将累积沙盒供给延迟定义为会话内每次工具调用所产生的沙盒初始化延迟之和，仅关注环境建立开销，排除沙盒内计算时间。

Fig. 9 shows the distribution of cumulative latency across sessions. The On-demand baseline exhibits steadily increasing latency over longer execution horizons due to repeated cold-start overheads, resulting in a heavy-tailed distribution. In contrast, SpecBox significantly reduces cumulative latency and maintains a much tighter distribution concentrated in the sub-second range. Compared to the On-demand baseline, SpecBox achieves a 4.53× reduction in cumulative sandbox provisioning latency. Against the Reserved baseline, SpecBox remains within a 10.6% performance gap while avoiding the substantial resource overhead of persistent sandbox allocation. These results demonstrate that predictive orchestration can effectively approximate Reserved execution performance under a On-demand deployment model.

图 9 展示了跨会话的累积延迟分布。按需基线由于反复的冷启动开销，在更长执行视野上延迟稳步增加，形成重尾分布。相比之下，SpecBox 显著降低累积延迟，并保持集中在亚秒区间的更紧分布。相对 按需 基线，SpecBox 将累积沙盒供给延迟降低 4.53×。相对 预留 基线，SpecBox 的性能差距保持在 10.6% 以内，同时避免了持久沙盒分配的巨大资源开销。这些结果表明，预测性编排能在 按需 部署模型下有效逼近 预留 执行性能。

5.2.2 Scalability / 可扩展性

Fig. 10(a) summarizes the scalability of the three runtimes under different concurrency, with QPS increased from 1 to 20. Across low-to-mid concurrency levels, SpecBox consistently remains a latency profile close to Reserved while reducing end-to-end delay compared with On-demand. At higher concurrency, the advantage over On-demand remains substantial: at QPS=20, Laplace achieves 88.7s P99 E2E latency, a 2.9× speedup over On-demand (257.2s). This indicates that SpecBox can absorb increasing concurrency without inheriting the long-tail latency explosion of the On-demand baseline.

图 10(a) 总结了三种运行时在不同并发下的可扩展性，QPS 从 1 增至 20。在低到中等并发水平，SpecBox 始终保持接近 预留 的延迟曲线，同时相对 按需 降低端到端延迟。在更高并发下，相对 按需 的优势依然显著：在 QPS=20 时，Laplace 达到 88.7s 的 P99 端到端延迟，相对 按需（257.2s）加速 2.9×。这表明 SpecBox 能够吸收不断增长的并发，而不会继承 按需 基线的长尾延迟爆炸。

Fig. 10(b) also reveals that the main scalability bottleneck of On-demand lies in cumulative sandbox provisioning. In the low-QPS regime, the mean cumulative sandbox provisioning latency stays below 4 seconds for all three modes. However, once the load reaches the higher-QPS regime (QPS≥5), On-demand suffers growing degradation from network contention and resource constraints, and its cumulative sandbox provisioning latency rises steadily and eventually surpasses 50 seconds. By contrast, Laplace remains tightly bounded in the sub-5-second range and continues to slightly outperform Reserved in provisioning latency, reflecting the benefit of SpecBox’s intent-aware sandbox prewarming and stochastic sandbox prefetching under concurrency pressure.

图 10(b) 还揭示，按需 的主要可扩展性瓶颈在于累积沙盒供给。在低 QPS 区间，三种模式的平均累积沙盒供给延迟均低于 4 秒。然而，一旦负载进入更高 QPS 区间（QPS≥5），按需 因网络争用与资源约束而持续劣化，其累积沙盒供给延迟稳步上升并最终超过 50 秒。相比之下，Laplace 被严格限制在 5 秒以下区间，并在供给延迟上继续略优于 预留，反映出 SpecBox 的意图感知沙盒预热与随机沙盒预取在并发压力下的收益。

5.2.3 Resource Footprint and Efficiency / 资源占用与效率

Fig. 10(c) shows that the CPU footprint of the three runtimes remains relatively compact under low-load conditions but diverges significantly as concurrency scales. While SpecBox exhibits no obvious advantages under low QPS loads, its strengths gradually emerge in the high-QPS regime where QPS≥8. It consistently restrains peak CPU utilization across all tested loads, with the peak resource consumption capped at around 12.2 cores. This translates to a 22.8%–23.3% reduction in peak CPU pressure compared to both On-demand and Reserved, which demand roughly 15.8–15.9 cores. Particularly in the high-QPS regime, SpecBox stabilizes within a tight band of ∼11–12 cores, whereas the two baselines fluctuate at a higher plateau of 14–16 cores, demonstrating that SpecBox effectively mitigates CPU contention (saving up to 25% CPU resource under peak load) without sacrificing sub-millisecond control-plane responsiveness.

图 10(c) 显示，三种运行时的 CPU 占用在低负载下相对紧凑，但随并发扩展而显著分化。虽然 SpecBox 在低 QPS 负载下没有明显优势，其长处在 QPS≥8 的高 QPS 区间逐渐显现。它在所有测试负载下持续抑制峰值 CPU 利用率，峰值资源消耗约封顶在 12.2 核。相对 按需 与 预留（约需 15.8–15.9 核），这相当于峰值 CPU 压力降低 22.8%–23.3%。尤其在高 QPS 区间，SpecBox 稳定在约 11–12 核的紧区间，而两个基线在 14–16 核的更高平台上波动，表明 SpecBox 有效缓解了 CPU 争用（峰值负载下最多节省 25% CPU 资源），同时不牺牲亚毫秒级控制面响应性。

The memory footprint (Fig. 10(d)) exhibits a pronounced resource separation across the evaluation spectrum. As expected, Reserved is the most memory-intensive variant due to its persistence strategy; its memory consumption surges from an initial 24.6 GiB at light load to a massive peak of 80.6 GiB, staying sustained above 60 GiB as concurrency intensifies. Conversely, On-demand maintains a lean profile, bounding its peak usage between 24.3 GiB and 40.4 GiB. Notably, SpecBox closely mirrors the trajectory of On-demand across the majority of QPS configurations, topping out at 49.4 GiB. This represents a 45.9% reduction in peak memory footprint compared to Reserved, demonstrating that SpecBox successfully eliminates the prohibitive memory holding costs of long-run sandboxes, while maintaining a predictable, serverless-like resource elasticity under heavy concurrent workflows.

内存占用（图 10(d)）在评估谱上呈现出显著的资源分离。如预期，预留 因其持久化策略是内存最密集的变体；其内存消耗从轻负载时的初始 24.6 GiB 飙升至高达 80.6 GiB 的峰值，并在并发加剧时持续高于 60 GiB。相反，按需 保持精简画像，峰值使用介于 24.3 GiB 与 40.4 GiB 之间。值得注意的是，SpecBox 在大多数 QPS 配置下紧密镜像 按需 的轨迹，最高达 49.4 GiB。相对 预留，这代表峰值内存占用降低 45.9%，表明 SpecBox 成功消除了长运行沙盒过高的内存占用成本，同时在重度并发工作流下保持可预测的、类无服务器的资源弹性。

5.3 Micro-Benchmarking / 微基准测试

This section provides mechanism-level attribution for the end-to-end improvements reported above. In particular, we isolate the effects of intent-aware sandbox prewarming, stochastic sandbox prefetching, semantic caching, and out-of-band data transmission to avoid conflating component contributions in the E2E section.

本节为上文报告的端到端改善提供机制级归因。具体而言，我们隔离意图感知沙盒预热、随机沙盒预取、语义缓存与带外数据传输的效果，以避免在 E2E 部分混淆各组件贡献。

5.3.1 Effectiveness of Intent-Aware Sandbox Prewarming / 意图感知沙盒预热的有效性

We further decompose intent-aware sandbox prewarming into three orthogonal factors: the keyword trigger threshold, the semantic model family, and the hybrid fusion policy. All results are measured on the same sampled tasks and repeated 100 times.

我们进一步将意图感知沙盒预热分解为三个正交因素：关键词触发阈值、语义模型族，以及混合融合策略。全部结果在相同采样任务上测量，并重复 100 次。

Sensitivity of Keyword Router Threshold γ. / 关键词路由器阈值 γ 的敏感性。

To rigorously isolate the impact of the keyword matching threshold, we evaluate γ∈{1,2,3} under identical workload profiles, executing 100 independent trials for each configuration to ensure statistical convergence. As quantified in Table 1, γ=2 emerges as the optimal configuration for the Keyword Router. Specifically, γ=2 substantially compresses the average keyword-driven waiting latency to 323.0 ms (a 2.43× reduction compared to 786.6 ms under γ=1) while maintaining a high predictive routing coverage of 95%.

为严格隔离关键词匹配阈值的影响，我们在相同工作负载画像下评估 γ∈{1,2,3}，每种配置执行 100 次独立试验以确保统计收敛。如表 1 所量化，γ=2 成为关键词路由器的最优配置。具体而言，γ=2 将关键词驱动的平均等待延迟大幅压缩至 323.0 ms（相对 γ=1 下的 786.6 ms 降低 2.43×），同时保持 95% 的高预测路由覆盖率。

The severe latency degradation under γ=1 is primarily driven by its hyper-sensitivity, which triggers 20 distinct tool mismatch instances across the 100 runs; these false positives inadvertently amplify cold-start fallback penalties and introduce heavy tail latencies. Conversely, while increasing the threshold to γ=3 further sharpens routing precision (yielding a 97% match rate with only 3 mismatch instances), it severely erodes the prewarm lead time, causing the average waiting time to climb back to 621.8 ms. This empirical trade-off confirms that γ=2 effectively balances early predictive agility with resource stability.

γ=1 下的严重延迟劣化主要由其超敏感性驱动，在 100 次运行中触发 20 次不同的工具不匹配；这些假阳性无意中放大了冷启动回退惩罚，并引入重尾延迟。相反，将阈值提高到 γ=3 进一步锐化路由精度（匹配率 97%，仅 3 次不匹配），但严重侵蚀预热领先时间，使平均等待时间回升至 621.8 ms。这一经验权衡确认，γ=2 有效平衡了早期预测敏捷性与资源稳定性。

EN Table 1: Keyword threshold sensitivity (γ=1,2,3). CN 表 1: 关键词阈值敏感性（γ=1,2,3）。

γ Avg Wait (ms) Match Rate Mismatch Runs
1 786.619 80.0% 20
2 323.021 95.0% 5
3 621.812 97.0% 3
Sensitivity of Semantic Router Models / 语义路由器模型的敏感性

Table 2 compares the performance and computational trade-offs of three semantic routing models under their optimal configurations:

表 2 比较了三种语义路由模型在各自最优配置下的性能与计算权衡：

  • Retrieval: This model is implemented as a non-neural, sparse token-level retriever. It leverages TF-IDF (Sparck Jones, 1972) weighting combined with a Top-N nearest neighbor aggregation network, constructing composite vector features across token unigrams, selective bigrams, and character n-grams (ranging from 3 to 5 characters).

  • Retrieval： 该模型实现为非神经、稀疏的 Token 级检索器。它利用 TF-IDF（Sparck Jones, 1972）加权，结合 Top-N 近邻聚合网络，跨 Token 一元组、选择性二元组与字符 n-gram（3 到 5 个字符）构造复合向量特征。

  • Encoder: This neural model is built upon a fine-tuned all-MiniLM-L6-v2 (sentence-transformers, 2026) transformer sequence representation network. It maps fluid multi-turn trajectories into dense vector spaces via contrastive pair-wise optimization under a CosineSimilarityLoss constraint.

  • Encoder： 该神经模型基于微调的 all-MiniLM-L6-v2（sentence-transformers, 2026）Transformer 序列表示网络。它通过 CosineSimilarityLoss 约束下的对比成对优化，将流动的多轮轨迹映射到稠密向量空间。

  • FastText: This lightweight model (Joulin et al., 2017) represents text as the average of word and subword embeddings followed by a linear classifier.

  • FastText： 该轻量模型（Joulin et al., 2017）将文本表示为词与子词嵌入的平均，后接线性分类器。

Empirically, as quantified in Table 2, both Retrieval and Encoder models achieve a optimal prewarming Hit Rate, demonstrating that both sparse lexical tokens and dense semantic features can successfully capture all necessary tool invocation targets. However, Retrieval achieves higher orchestration quality and routing precision, yielding a Micro-F1 of 0.970 and a Precision of 0.942, whereas Encoder degrades to a Micro-F1 of 0.776 and a Precision of 0.634.

经验上，如表 2 所量化，Retrieval 与 Encoder 模型均达到最优预热命中率，表明稀疏词汇 Token 与稠密语义特征都能成功捕捉全部必要的工具调用目标。然而，Retrieval 获得更高的编排质量与路由精度，Micro-F1 为 0.970、Precision 为 0.942，而 Encoder 降至 Micro-F1 0.776、Precision 0.634。

Furthermore, from an runtime efficiency perspective, Retrieval operates with a significantly lower computational footprint, executing with an average inference latency of only 2.116 ms–yielding a 3.70× speedup compared to the 7.822 ms latency incurred by Encoder. Conversely, while FastText provides ultra-low execution latency (0.171 ms), its shallow token representation space fails to deliver usable discrimination under multi-turn reasoning context drifts, resulting in a Micro-F1 and Hit Rate of merely 0.124. Consequently, Retrieval is selected as the production instance for Laplace’s semantic branch, as it strikes the optimal Pareto-efficiency frontier between high-fidelity prediction accuracy and low-overhead control-plane latency.

此外，从运行时效率角度看，Retrieval 的计算占用显著更低，平均推理延迟仅 2.116 ms——相对 Encoder 的 7.822 ms 延迟加速 3.70×。相反，虽然 FastText 提供超低执行延迟（0.171 ms），其浅层 Token 表示空间在多轮推理上下文漂移下无法提供可用判别力，Micro-F1 与命中率仅为 0.124。因此，Retrieval 被选为 Laplace 语义分支的生产实例，因为它在高保真预测精度与低开销控制面延迟之间达到最优帕累托效率前沿。

EN Table 2: Semantic router model comparison (best operating point per model). CN 表 2: 语义路由器模型比较（每种模型的最佳工作点）。

Model Micro-F1 Hit Rate (Top-3) Precision Avg Latency (ms)
Retrieval 0.970 0.992 0.942 2.116
Encoder 0.776 0.968 0.634 7.822
FastText 0.124 0.124 0.124 0.171
Sensitivity of Assembly Policies. / 组装策略的敏感性。

To rigorously isolate the algorithmic impact of the intent combination layer, we evaluate three distinct fusion policy paradigms under identical multi-turn context distributions:

为严格隔离意图组合层的算法影响，我们在相同多轮上下文分布下评估三种不同的融合策略范式：

  • Union (∪): The predictive prewarming primitive is non-blocking and asynchronously dispatched the exact microsecond either the keyword router or the semantic router hits their respective individual activation boundaries.

  • 并集（∪）： 预测性预热原语是非阻塞的，在关键词路由器或语义路由器命中各自激活边界的精确微秒时刻异步分派。

  • Intersection (∩): Predictive orchestration strictly enforces a dual-router consensus; a sandbox is prewarmed if and only if both the keyword router and semantic router concurrently validate the sandbox candidate’s invocation intent.

  • 交集（∩）： 预测性编排严格强制双路由器共识；当且仅当关键词路由器与语义路由器同时验证该沙盒候选的调用意图时，才预热沙盒。

  • Weighted: This hybrid policy aggregates multi-modal intent metrics into a centralized candidate score: s⁡(c)=wk⋅sk(c)+ws⋅ss(c), where sk(c) denotes the matched-keyword ratio (normalized against the static lexicon scale) and ss(c) represents the real-time semantic retrieval similarity vector. A predictive trigger is dispatched only when clearing a rigid threshold: s⁡(c)≥τf. In our empirical runner, we implement a symmetric baseline configuration with wk=0.5,ws=0.5, and τf=0.5.

  • 加权： 该混合策略将多模态意图度量聚合为中心化候选分数：s(c)=wk·sk(c)+ws·ss(c)，其中 sk(c) 表示匹配关键词比率（相对静态词表规模归一化），ss(c) 表示实时语义检索相似性向量。仅当越过刚性阈值 s(c)≥τf 时才分派预测触发。在我们的经验运行器中，我们实现对称基线配置：wk=0.5，ws=0.5，τf=0.5。

As quantified in Table 3, there is a clear trade-off between the aggressive Union Assembly and the conservative Intersection policy. Union Assembly prioritizes latency-masking agility, reducing the average waiting latency to just 124.45 ms at the cost of a slight 5.0% cold-start ratio. In contrast, the conservative Intersection policy eliminates cold starts (0.0%) by enforcing strict dual-router consensus, but at the cost of blocking the critical path and driving the average latency up to 393.52 ms (3.16× higher than Union).

如表 3 所量化，激进的 并集组装 与保守的 交集 策略之间存在清晰权衡。并集组装 优先延迟掩盖敏捷性，将平均等待延迟降至仅 124.45 ms，代价是轻微的 5.0% 冷启动比率。相反，保守的 交集 策略通过强制严格双路由器共识消除冷启动（0.0%），但代价是阻塞关键路径，并将平均延迟推高至 393.52 ms（比并集高 3.16×）。

The Weighted policy performs the worst, inflating latency to 1308.38 ms due to a 45.0% cold-start ratio. This failure stems from the structural misalignment between keyword tokens and high-dimensional semantics. Without dynamic re-normalization, shifting context distributions under multi-turn reasoning cause the combined scores to drift, frequently failing to clear the static τf=0.5 threshold. These results validate our choice of a Union-first assembly design to maximize latency-masking performance while maintaining practical routing correctness.

加权 策略表现最差，因 45.0% 的冷启动比率将延迟膨胀至 1308.38 ms。这一失败源于关键词 Token 与高维语义之间的结构错位。在没有动态重归一化的情况下，多轮推理下移动的上下文分布使组合分数漂移，频繁无法越过静态 τf=0.5 阈值。这些结果验证了我们选择 并集优先 组装设计，以最大化延迟掩盖性能，同时保持实用的路由正确性。

EN Table 3: Assembly policy trade-off. CN 表 3: 组装策略权衡。

Assembly Mode Avg Wait (ms) Target Match Rate Cold Start Ratio
Union 124.446 95.0% 5.0%
Intersection 393.523 100.0% 0.0%
Weighted 1308.378 55.0% 45.0%

5.3.2 Effectiveness of Stochastic Sandbox Prefetching / 随机沙盒预取的有效性

EN Figure 11: Dynamic execution profiling across a 10-turn conversation horizon, benchmarking per-turn average waiting latency (left) against the corresponding average cold-start sandbox activation count (right). CN 图 11: 跨 10 轮对话视野的动态执行剖析，对比每轮平均等待延迟（左）与对应的平均冷启动沙盒激活次数（右）。

To isolate the contribution of stochastic prefetching under multi-turn agent execution, we evaluate Laplace with two deployment variants under an identical planning workload:

为在多轮智能体执行下隔离随机预取的贡献，我们在相同规划工作负载下评估 Laplace 的两种部署变体：

  • SpecBox-Reactive: A routing-only baseline that performs inline token-level intent detection but does not use cross-step transition prediction.

  • SpecBox-Reactive： 仅路由的基线，执行内联 Token 级意图检测，但不使用跨步转移预测。

  • SpecBox-Proactive: The full design that couples online hybrid routing with a stochastic Markovian prefetcher over the sandbox dependency graph (SDG), enabling asynchronous sandbox preparation before the next step.

  • SpecBox-Proactive： 完整设计，将在线混合路由与沙盒依赖图（SDG）上的随机马尔可夫预取器耦合，使下一步之前即可异步准备沙盒。

Fig. 11 reports per-turn average waiting latency and cold-start counts over a 10-turn horizon. During Turn 1, both variants show similar latency (512.06 ms vs. 540.06 ms), since no historical transition signal is available to initialize the Markov predictor. From Turn 2 onward, the two trajectories diverge sharply. SpecBox*-Proactive* reduces average waiting latency from 540.06 ms to 138.14 ms at Turn 2 and further to 97.14 ms by Turn 10, while SpecBox*-Reactive* increases to 583.26 ms at Turn 10. This corresponds to a 6.0× end-horizon latency reduction. The mechanism is consistent with the cold-start telemetry. Under SpecBox*-Reactive*, average cold-start count peaks at 2.90 instances per turn (Turn 9), indicating repeated exposure to sandbox initialization cost on the critical path. In contrast, SpecBox*-Proactive* keeps per-turn cold starts within 0.24–0.83, effectively masking startup latency through early scheduling. These results show that Laplace improves multi-turn responsiveness through accurate, low-overhead temporal prefetching rather than infrastructure over-provisioning.

图 11 报告 10 轮视野上的每轮平均等待延迟与冷启动次数。在第 1 轮，两种变体延迟相近（512.06 ms vs. 540.06 ms），因为尚无历史转移信号可用于初始化马尔可夫预测器。从第 2 轮起，两条轨迹急剧分化。SpecBox-Proactive 将平均等待延迟从第 2 轮的 540.06 ms 降至 138.14 ms，并在第 10 轮进一步降至 97.14 ms，而 SpecBox-Reactive 在第 10 轮升至 583.26 ms。这对应 6.0× 的视野末端延迟降低。该机制与冷启动遥测一致。在 SpecBox-Reactive 下，平均冷启动次数在第 9 轮达到每轮 2.90 次峰值，表明关键路径上反复暴露于沙盒初始化成本。相反，SpecBox-Proactive 将每轮冷启动保持在 0.24–0.83，通过早期调度有效掩盖启动延迟。这些结果表明，Laplace 通过准确、低开销的时间预取而非基础设施过度供给，改善了多轮响应性。

5.3.3 Effectiveness of Semantic Cache / 语义缓存的有效性

To quantify the impact of our semantic cache, we benchmark three variants under the same repeated-request workload over 100 runs:

为量化语义缓存的影响，我们在相同重复请求工作负载上、100 次运行中基准测试三种变体：

  • No Cache: that executes every repeated request from scratch.

  • 无缓存： 每次重复请求都从零执行。

  • Exact-Match Cache: A strict cache that matches on tool identity and normalized argument equality.

  • 精确匹配缓存： 严格缓存，按工具身份与规范化参数相等进行匹配。

  • Semantic Cache: reusing results when the tool identity matches and semantic similarity exceeds the threshold.

  • 语义缓存： 当工具身份匹配且语义相似性超过阈值时复用结果。

EN Table 4: Semantic cache trade-off. CN 表 4: 语义缓存权衡。

Setting Avg Wait (ms) Hit Rate Bypass Ratio
No Cache 412.78 0.0% 0.0%
Exact-Match Cache 233.64 33.6% 100.0%
Semantic Cache (τ=0.6) 141.93 37.4% 84.8%

As shown in Table 4, while exact-match caching improves performance over No Cache, semantic caching captures a wider envelope of near-duplicate requests by tolerating surface-form variations. Compared to No Cache, semantic caching achieves a 2.91× speedup in average waiting latency (dropping from 412.78 ms to 141.93 ms) and increases the cache hit rate to 37.4% .

如表 4 所示，虽然精确匹配缓存相对 无缓存 改善了性能，语义缓存通过容忍表层形式变化捕捉了更宽的近重复请求包络。相对 无缓存，语义缓存将平均等待延迟加速 2.91×（从 412.78 ms 降至 141.93 ms），并将缓存命中率提升至 37.4%。

However, sematic caching is slightly lower than exact matching because a small fraction of semantic hits are not sufficiently reliable to skip sandbox setup and must fall back to normal execution after validation. This behavior is consistent with the intended design: semantic equivalence expands the reusable request space, but some loose matches still require conservative verification before execution can be bypassed.

然而，语义缓存的旁路比率略低于精确匹配，因为一小部分语义命中尚不足够可靠以跳过沙盒建立，必须在验证后回退到正常执行。这一行为与预期设计一致：语义等价扩大可复用请求空间，但某些宽松匹配在旁路执行之前仍需保守验证。

5.3.4 Effectiveness of Out-of-Band Data Transmission / 带外数据传输的有效性

EN Figure 12: Data transmission latency scaling profiles under varying payload sizes, comparing SpecBox’s out-of-band data transmission against standard JSON-RPC serialization. CN 图 12: 不同载荷大小下的数据传输延迟扩展画像，比较 SpecBox 的带外数据传输与标准 JSON-RPC 序列化。

To evaluate the architectural efficiency of our control-and-data plane separation, we isolate the data transmission overhead by benchmarking two distinct transport paradigms across an exponential data payload spectrum scaling from 1.00 MB to 1000.00 MB:

为评估控制面与数据面分离的架构效率，我们通过在从 1.00 MB 到 1000.00 MB 指数扩展的数据载荷谱上基准测试两种不同传输范式，隔离数据传输开销：

  • Out-of-Band: Our proposed out-of-band transport mechanism. It completely bypasses the control-plane RPC tunnel by writing bulky state payloads directly to a dedicated local shared-memory substrate or zero-copy virtualized host-guest ring buffers, passing only lightweight, fixed-size references over the wire.

  • 带外： 我们提出的带外传输机制。它通过将大宗状态载荷直接写入专用本地共享内存基底或零拷贝虚拟化主机—客户机环形缓冲区，完全绕过控制面 RPC 隧道，仅在线路上传递轻量、固定大小的引用。

  • JSON-RPC: The standard baseline paradigm utilized in conventional agent runtimes. It marshals multi-modal data payloads directly into the inline runtime execution stream, forcing the control-plane to serialize and transport raw data matrices synchronously via text-based JSON-RPC network primitives.

  • JSON-RPC： 传统智能体运行时使用的标准基线范式。它将多模态数据载荷直接编组到内联运行时执行流中，迫使控制面通过基于文本的 JSON-RPC 网络原语同步序列化并传输原始数据矩阵。

As shown in Fig. 12, the empirical measurements reveal a clear scaling divergence between the two protocols across an exponential payload spectrum. At the baseline threshold (1.00 MB), both configurations demonstrate sub-3 millisecond performance, with Out-of-Band maintaining a slight edge (1.95 ms vs. 2.28 ms). However, as the payload expands, the JSON-RPC pipeline suffers a catastrophic linear performance degeneration; throttled by heavy serialization bottlenecks, its latency grows to 132.45 ms at 100.00 MB and reaches 1873.16 ms at the 1000.00 MB boundary. Conversely, Out-of-Band demonstrates a near-constant O⁡(1) scaling profile, drifting to only 5.97 ms at the 1 GB boundary—a 313.55× latency reduction.

如图 12 所示，经验测量揭示了两种协议在指数载荷谱上的清晰扩展分化。在基线阈值（1.00 MB），两种配置均展现亚 3 毫秒性能，带外 保持轻微优势（1.95 ms vs. 2.28 ms）。然而，随着载荷扩大，JSON-RPC 流水线遭受灾难性的线性性能退化；受沉重序列化瓶颈节流，其延迟在 100.00 MB 增至 132.45 ms，并在 1000.00 MB 边界达到 1873.16 ms。相反，带外 展现近恒定的 O(1) 扩展画像，在 1 GB 边界仅漂移至 5.97 ms——延迟降低 313.55×。

This performance gap stems from eliminating critical-path serialization and memory copying. In multi-turn workflows, agents frequently exchange large multi-modal states like high-dimensional vectors or media files. Standard JSON-RPC stalls the control loop due to synchronous serialization, whereas SpecBox’s out-of-band design decouples control signals from raw payload routing, reducing data transfer to a constant-time reference-passing operation. These results demonstrate that separating the control and data planes keeps SpecBox’s orchestration overhead minimal and independent of payload size.

这一性能差距源于消除了关键路径上的序列化与内存拷贝。在多轮工作流中，智能体频繁交换大型多模态状态，如高维向量或媒体文件。标准 JSON-RPC 因同步序列化而使控制循环停滞，而 SpecBox 的带外设计将控制信号与原始载荷路由解耦，将数据传输简化为常数时间的引用传递操作。这些结果表明，分离控制面与数据面使 SpecBox 的编排开销保持最小，且与载荷大小无关。

6 Discussion / 讨论

Architectural Overhead. A critical concern in predictive serving runtimes is whether the system-level orchestration mechanisms introduce non-negligible processing penalties onto the critical path. In SpecBox, this control-plane overhead is thoroughly isolated from the GPU-bound inference loop through strict out-of-band execution and parallel routing mechanics. While the token-level scanning runs with deterministic 𝒪⁡(1) complexity, the inherently heavier semantic retrieval engine is offloaded to dedicated background CPU worker threads. Because the intent router utilizes an Union Assembly (∪), the critical path never blocks for late-arriving semantic embeddings; any early fast-path trigger instantly dispatches the activation primitive within microseconds. Combined with the prefetching daemon–which evaluates low-dimension Markov matrix transitions bounded by the small cardinality of active sandboxes–SpecBox restricts its control-plane telemetry to tens of microseconds, securing a nearly zero-cost latency impact on token generation.

架构开销。 预测性服务运行时的一个关键关切是：系统级编排机制是否会在关键路径上引入不可忽视的处理惩罚。在 SpecBox 中，这一控制面开销通过严格的带外执行与并行路由机制，与 GPU 绑定的推理循环彻底隔离。Token 级扫描以确定性 𝒪(1) 复杂度运行，而固有更重的语义检索引擎被卸载到专用后台 CPU 工作线程。由于意图路由器采用 并集组装（∪），关键路径从不等待迟到的语义嵌入；任何早期快速路径触发都会在微秒内立即分派激活原语。结合预取守护进程——它评估由活跃沙盒小基数所界定的低维马尔可夫矩阵转移——SpecBox 将其控制面遥测限制在数十微秒，从而对 Token 生成实现近乎零成本的延迟影响。

Ecosystem Generalization. We position SpecBox as a runtime middleware layer operating between upstream reasoning agents and downstream execution environments. This design leverages two widely available capabilities in modern agent ecosystems (Wu et al., 2023; Langchain-ai, 2026; Gao et al., 2024): streaming token-level outputs from upstream frameworks, and standardized tool interfaces provided by protocols such as MCP (Anthropic PBC, 2024). As a result, SpecBox requires no additional system-level constraints beyond what is already supported in existing agent and sandbox orchestration stacks.

生态泛化。 我们将 SpecBox 定位为运行在上游推理智能体与下游执行环境之间的运行时中间件层。这一设计利用现代智能体生态中两项广泛可用的能力（Wu et al., 2023；Langchain-ai, 2026；Gao et al., 2024）：上游框架的流式 Token 级输出，以及 MCP（Anthropic PBC, 2024）等协议提供的标准化工具接口。因此，SpecBox 除现有智能体与沙盒编排栈已支持的能力外，不要求额外的系统级约束。

Importantly, this execution abstraction extends beyond inference-time serving to agentic reinforcement learning (Agentic RL) frameworks, such as VERL (Sheng et al., 2024) and Slime (Zhu et al., 2025), where rollout generation and environment interaction follow a structurally similar loop. In these Agentic RL settings, SpecBox ’s runtime optimizations—namely intent-aware sandbox prewarming and stochastic sandbox prefetching—can be seamlessly adapted to mask environment initialization and interaction overheads during training rollouts with modest integration effort.

重要的是，这一执行抽象可从推理时服务延伸到智能体强化学习（Agentic RL）框架，如 VERL（Sheng et al., 2024）与 Slime（Zhu et al., 2025），其中 rollout 生成与环境交互遵循结构相似的循环。在这些 Agentic RL 设置中，SpecBox 的运行时优化——即 意图感知沙盒预热 与 随机沙盒预取——可以较小的集成代价无缝适配，以在训练 rollout 期间掩盖环境初始化与交互开销。

Limitations and Future Work. Despite its performance gains, SpecBox exhibits certain architectural boundaries. On the control plane, our prefetcher assumes first-order history dependence via a Markov Chain model, which may suffer from diminished prediction accuracy during open-ended, long-horizon agent workflows. Crucially, SpecBox inherently mitigates this via its hierarchical, two-tiered preparation: any inter-step prefetch miss gracefully falls back to the intra-step intent-aware prewarming during real-time token streaming. This design guarantees that worst-case environment latency remains tightly bounded within the LLM’s decoding phase. Future work will investigate online Graph Neural Networks (GNNs) to adaptively capture non-linear transition patterns.

局限与未来工作。 尽管取得了性能收益，SpecBox 仍表现出某些架构边界。在控制面上，我们的预取器经由马尔可夫链模型假设一阶历史依赖，这在开放式、长时程智能体工作流中可能遭受预测精度下降。关键的是，SpecBox 通过其分层、两级准备内在地缓解了这一点：任何跨步预取未命中都会优雅回退到实时 Token 流期间的步内意图感知预热。这一设计保证最坏情况下的环境延迟被严格限制在 LLM 解码阶段之内。未来工作将研究在线图神经网络（GNN），以自适应捕捉非线性转移模式。

On the data plane, our mmap-based transmission requires co-locating the Agent Engine and sandboxes within the same host boundary. While this design seamlessly aligns with mainstream multi-tenant serving topologies (e.g., Kubernetes Pod IPC sharing or Sidecar patterns) and leverages robust container-level namespace and cgroup security isolation, it restricts single-session cross-node scaling. To support large-scale distributed clusters, we plan to integrate RDMA-assisted zero-copy transmission, extending our out-of-band data plane into disaggregated cloud infrastructures.

在数据面上，我们基于 mmap 的传输要求将智能体引擎与沙盒共置在同一主机边界内。虽然这一设计与主流多租户服务拓扑无缝对齐（例如 Kubernetes Pod IPC 共享或 Sidecar 模式），并利用稳健的容器级命名空间与 cgroup 安全隔离，它限制了单会话跨节点扩展。为支持大规模分布式集群，我们计划集成 RDMA 辅助的零拷贝传输，将带外数据面延伸到解耦的云基础设施中。

7 Related Work / 相关工作

Lightweight Sandbox Runtimes. The systems community has actively explored lightweight isolation mechanisms to mitigate the physical instantiation cost of serverless execution environments. Notable advancements include micro-virtual machines (MicroVMs) like Firecracker (Agache et al., 2020) and RunD (Li et al., 2022), WebAssembly (Wasm) (WebAssembly, 2026) runtimes, and process-level snapshotting/forking frameworks such as FaaSnap (Ao et al., 2022) and TrEnv-X (Huang et al., 2026). These infrastructure-level systems focus on minimizing localized host setup or exploiting hardware-assisted remote memory pools (e.g., CXL/RDMA) to share and reuse physical sandboxes across tenants.

轻量沙盒运行时。 系统社区已积极探索轻量隔离机制，以缓解无服务器执行环境的物理实例化成本。显著进展包括 Firecracker（Agache et al., 2020）与 RunD（Li et al., 2022）等微虚拟机（MicroVM）、WebAssembly（Wasm）（WebAssembly, 2026）运行时，以及 FaaSnap（Ao et al., 2022）与 TrEnv-X（Huang et al., 2026）等进程级快照/分叉框架。这些基础设施级系统聚焦于最小化局部主机建立，或利用硬件辅助的远程内存池（如 CXL/RDMA）跨租户共享并复用物理沙盒。

Crucially, these low-layer container optimizations are entirely orthogonal to SpecBox. While microVM snapshots or repurposable sandboxes successfully compress the physical infrastructure boot time to milliseconds, application-layer handshakes (such as MCP discovery hooks) and environment attachment bottlenecks still natively linger on the critical runtime path. SpecBox operates at a higher, application-perceptive orchestration layer; it complements these physical speedups by exploiting a distinct temporal dimension—overlapping control-plane scheduling with streaming token generation—to fully mask, rather than compress, the intrinsic readiness latency.

关键的是，这些低层容器优化与 SpecBox 完全正交。虽然微虚拟机快照或可再利用沙盒成功将物理基础设施启动时间压缩到毫秒级，应用层握手（如 MCP 发现钩子）与环境附着瓶颈仍会原生滞留在关键运行时路径上。SpecBox 运行在更高的、应用感知的编排层；它通过利用一个不同的时间维度——将控制面调度与流式 Token 生成重叠——来补充这些物理加速，从而完全掩盖而非压缩固有的就绪延迟。

Serverless Predictive Prewarming. Predictive prewarming is a widely embraced technique in standard serverless frameworks to eradicate the notorious tail-latency penalties of tenant cold starts. State-of-the-art prefetching daemons (e.g., Mitosis (Wei et al., 2023) and IceBreaker (Roy et al., 2022)) primarily depend on historical time-series analytics, statistical invocation frequency histograms, or temporal correlation clustering to predictively prepare idle container runtimes.

无服务器预测性预热。 预测性预热是标准无服务器框架中被广泛采用的技术，用以消除租户冷启动臭名昭著的尾延迟惩罚。最先进的预取守护进程（如 Mitosis（Wei et al., 2023）与 IceBreaker（Roy et al., 2022））主要依赖历史时间序列分析、统计调用频率直方图，或时间相关聚类，以预测性地准备空闲容器运行时。

However, these classical prewarming paradigms inherently assume that incoming requests follow independent and identically distributed arrival models or deterministic time-triggered patterns. This core assumption breaks down completely under emerging LLM Agent workloads. In multi-turn autonomous agent reasoning loops, tool invocations are dynamically, autonomously, and non-linearly determined by the LLM’s fluid context trajectory, presenting complex execution dependency horizons. SpecBox bridges this gap by co-designing the serving infrastructure with explicit agent behavioral characteristics, introducing a stochastic Markovian predictive framework built over a sandbox dependency graph (SDG) to model real-time autonomous state transitions.

然而，这些经典预热范式内在假设传入请求遵循独立同分布到达模型或确定性时间触发模式。这一核心假设在新兴 LLM 智能体工作负载下彻底失效。在多轮自主智能体推理循环中，工具调用由 LLM 流动的上下文轨迹动态、自主且非线性地决定，呈现出复杂的执行依赖视野。SpecBox 通过将服务基础设施与显式的智能体行为特征协同设计来弥合这一鸿沟，引入构建在沙盒依赖图（SDG）之上的随机马尔可夫预测框架，以建模实时自主状态转移。

LLM Serving and Agent Orchestration. Accelerating the end-to-end execution of Large Language Models has driven extensive research across the AI systems spectrum. On the model-serving boundary, mainstream runtimes such as vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024) optimize GPU-internal kernel execution, memory caching via PagedAttention, and automated speculative decoding. Parallel to this, application-level agent orchestration platforms like AgentScope (Gao et al., 2024), AutoGen (Wu et al., 2023) and LangGraph (Langchain-ai, 2026) provide modular abstractions for programming multi-turn autonomous multi-agent teams.

LLM 服务与智能体编排。 加速大语言模型的端到端执行，推动了 AI 系统谱系上的广泛研究。在模型服务边界，vLLM（Kwon et al., 2023）与 SGLang（Zheng et al., 2024）等主流运行时优化 GPU 内部内核执行、经由 PagedAttention 的内存缓存，以及自动化推测解码。与此并行，AgentScope（Gao et al., 2024）、AutoGen（Wu et al., 2023）与 LangGraph（Langchain-ai, 2026）等应用级智能体编排平台，为编程多轮自主多智能体团队提供模块化抽象。

Nevertheless, existing LLM serving engines fundamentally treat model execution as a self-contained GPU computing unit, entirely oblivious to the physical host-plane environment friction when interacting with external tools. Conversely, high-level agent frameworks lack system-level visibility into cloud-native host topologies, resulting in uncoordinated data transfer and execution stalls. SpecBox operates as a runtime middleware layer that bridges this gap, enabling co-optimized out-of-band control-plane predictive routing and shared-memory data transmission across heterogeneous multi-tenant host execution boundaries.

然而，现有 LLM 服务引擎从根本上将模型执行视为自包含的 GPU 计算单元，完全无视与外部工具交互时物理主机平面的环境摩擦。相反，高层智能体框架缺乏对云原生主机拓扑的系统级可见性，导致不协调的数据传输与执行停滞。SpecBox 作为运行时中间件层运作以弥合这一鸿沟，在异构多租户主机执行边界上实现协同优化的带外控制面预测路由与共享内存数据传输。

8 Conclusions / 结论

We present SpecBox, a predictive execution runtime for LLM-based agent systems that reduces tail latency and resource inefficiency in multi-tenant environments. By analyzing the end-to-end agent execution step, we find that inefficiencies primarily arise from rigid dependencies among reasoning, environment initialization, and sandbox execution. SpecBox addresses this through a unified design that enables temporal overlap between LLM execution and sandbox setup, anticipates future sandbox needs across steps to reduce cross-step cold-start overheads, and eliminates redundant computation and communication through reuse-aware execution and out-of-band data transport. SpecBox is implemented on top of AgentScope (Gao et al., 2024) and evaluated it under highly concurrent multi-turn workloads. Results show up to a 2.9× reduction in P99 latency and 45.9% lower peak memory usage, with a 97.9% prewarming hit rate. These results demonstrate that exploiting temporal overlap across the agent execution loop is an effective and general mechanism for improving efficiency in large-scale agent deployments.

我们提出 SpecBox，一个面向基于 LLM 的智能体系统的预测性执行运行时，用于降低多租户环境中的尾延迟与资源低效。通过分析端到端智能体执行步骤，我们发现低效主要源于推理、环境初始化与沙盒执行之间的僵硬依赖。SpecBox 通过统一设计加以应对：使 LLM 执行与沙盒建立在时间上重叠，跨步骤预判未来沙盒需求以降低跨步冷启动开销，并通过复用感知执行与带外数据传输消除冗余计算与通信。SpecBox 实现于 AgentScope（Gao et al., 2024）之上，并在高并发多轮工作负载下进行评估。结果显示 P99 延迟最多降低 2.9×，峰值内存使用降低 45.9%，预热命中率为 97.9%。这些结果表明，利用智能体执行循环中的时间重叠，是改善大规模智能体部署效率的有效且通用的机制。

3.5 - 论文分析

论文内容分析

SpecBox 论文解读

SpecBox 不是一篇“把容器启动得更快”的无服务器论文，也不是一篇“把 LLM 解码得更快”的推理系统论文。它要回答的是一个更窄、也更贴近当前智能体落地的问题：在工具沙盒已经通过 MCP 从智能体引擎解耦出去之后，如何在不永久预留全部沙盒的前提下，把环境准备从端到端关键路径上拿掉。

一句话概括其主张：智能体循环里真正昂贵的，往往不是沙盒启动本身，而是 AgentScope / AutoGen / LangGraph 这类运行时把“推理完成”和“环境就绪”串成一条硬依赖。SpecBox 用推测把这条依赖拆开——在 Token 还在生成时就开始预热当前步可能用到的沙盒，在当前工具还在跑时就开始预取下一步可能切换到的沙盒，再用语义缓存和带外传输把重复执行与大宗产物搬移从关键路径上削掉。

图 1 把这个主张画得很清楚：朴素路径是 GPU 先说完、CPU 再启动环境；SpecBox 的路径是两边重叠。后文的全部机制，都是在给这条重叠找一个既够早、又不太浪费的触发时机。

1. 它站在哪一层，解决哪一类张力

先把 SpecBox 放到智能体技术栈里，避免把它理解成通用调度器或通用冷启动优化。

用户请求
  → 智能体引擎（规划 / ReAct / 工具选择）     ← AgentScope 等
      → 流式 Token
  → SpecBox（本篇的位置：引擎与沙盒之间的运行时中间件）
      → 控制面：意图路由、马尔可夫预取、生命周期
      → 数据面：语义缓存、共享内存带外传输
  → MCP 工具沙盒（Playwright / Jupyter / Neo4j / PaperSearch …）

云原生把沙盒做成按需微服务之后，出现了一组经典两难：

部署方式 延迟 资源 规模化后果
预留（Reserved） 接近下限 空闲容器吃满主机内存/CPU 上千种细粒度 MCP 工具时经济上不可持续
按需（On-demand） 每次调用都可能秒级冷启动 精简 多轮会话里启动开销沿关键路径累加，P99 爆炸

论文强调：冷启动的物理成本（镜像、命名空间、文件系统、握手）确实存在，但默认运行时把这些成本百分之百暴露在关键路径上，才是交互式智能体服务变慢的主因。GPU 在生成 Token 时，CPU 侧环境是闲的；等 Token 结束、工具调用被完全确定，GPU 又要反过来等沙盒。两边互相空转。

这与同目录下 Aries 那篇“重新思考智能体云基础设施”的观察是互补的：Aries 指出工具沙盒大部分时间空闲、执行时突发，快照挂起又不划算；SpecBox 则给出一种服务期的折中——不要永久占着，也不要每次从零等，而是用推测把准备藏进已经在发生的计算里。

2. 为什么现成的预热、缓存、推测执行不够用

论文花了相当篇幅说明：无服务器领域的预热（Mitosis、IceBreaker）、工作流编排、容器快照（Firecracker、FaaSnap、TrEnv-X）以及 LLM 引擎里的推测解码（vLLM、SGLang），都不能直接搬过来。原因不在工程细节，而在智能体执行的生成方式：

  1. 执行目标在运行时才出现。 传统工作流图或 serverless 函数在请求到达前就知道要调谁。智能体的工具调用是自回归涌现的：不到 Token 流里积累足够语义，下一步调哪个 MCP 工具都不确定。
  2. 多轮轨迹依赖中间观测。 论文检索之后是否读文档、数据分析之后是否出图，取决于上一步结果，而不是固定 DAG。
  3. 意图信号是逐步变清晰的。 早期 Token 里充斥 read / get / search 这类跨工具复用的动词。预热越早，重叠窗口越大，假阳性也越多。

因此 SpecBox 的设计问题可以改写成三个互相打架的约束：

  • 步内：从尚不完整的 Token 流里尽早猜对工具，但不能把主机预热崩掉。
  • 跨步：在下一步意图出现前把后继沙盒准备好，但不能退化成“把所有可能后继都预留”。
  • 数据面：跳过重复工作和大宗传输，但不能改变智能体从沙盒里看到的语义，也不能拆掉 MCP 控制协议。

三项技术分别钉这三条约束，而且是分层回退关系，不是三个可有可无的插件。

图 2 把单步延迟拆成：上下文处理、Token 生成、环境准备 Tenv_prep、数据搬运 Tdata_io、沙盒执行 Tsandbox_exec。SpecBox 不动 LLM 推理栈，只打后三段。这是一个很强的工程立场：换引擎、换模型都不该要求改 SpecBox；代价是它也无法做更深的“解码–工具”联合调度（例如工具 JSON 一出现就截断解码、或对工具参数做推测填充）。

3. 机制一：意图感知预热——用“刚好够用的意图”换重叠窗口

3.1 核心操作

朴素运行时的时序是：

完整 tool_call 解析完成 → 开始拉起沙盒 → 握手 → 执行

SpecBox 改成：

Token 流正在生成
  ├─ 关键词路由器：微秒级扫描工具词表
  └─ 语义路由器：把当前上下文和工具意图表示做匹配
        ↓
  并集组装：任一可信信号即可通知沙盒管理器预热
        ↓
  与剩余 Tgeneration(N) 重叠

它要的不是完整、结构化的工具调用，而是 just enough intent：足够用来选定沙盒类型，不必等到参数和调用格式都生成完毕。图 3 里规划智能体把用户请求展开成 ReAct 步骤，关键词线索和任务级语义共同决定是否预热。

3.2 两个路由器为什么必须并存

这是全文最有系统设计味道的权衡，图 4 把它画成路由策略象限。

  • 关键词路由器快但不稳。 research / search / slide 出现在很多工具描述里。阈值 γ=1 会触发大量假阳性（100 次运行里 20 次工具不匹配），平均等待反而升到 786 ms，因为错误预热之后还要为真正的工具付冷启动。γ=3 匹配率到 97%，但预热领先时间被吃掉，等待回升到 622 ms。γ=2 是甜点：323 ms、95% 覆盖、5 次不匹配。
  • 语义路由器稳但晚。 它能抓住“措辞和工具词表几乎不重叠”的隐式意图，但可靠语义通常需要更长前缀。对 PaperSearch、Neo4j 这种冷启动可到约 20 秒的重量级沙盒，等语义路由器单独拍板，往往已经来不及把启动藏进剩余解码窗口。
  • 交集看起来更精确，实际更慢。 每次触发都要等更慢的那一路，并且任一路召回失败就会丢掉有效工具。
  • 并集是工程上的主选择。 每个路由器自己管假阳性，第一个可信信号就可以开工。实验里平均等待 124 ms，冷启动率 5%；交集把冷启动打到 0%，但等待变成 394 ms（高 3.16×）。加权融合最差（1308 ms、45% 冷启动），因为它把异构分数硬加到静态阈值上，多轮上下文一漂就跨不过 τf=0.5。

可以这样记：并集牺牲的是少量误预热内存，换回的是把冷启动从关键路径上抹掉的能力。 在交互式尾延迟优先的服务里，这个交换是合理的；在内存极度紧张、工具镜像非常重的集群里，5% 的假阳性仍可能不可接受，这时应把预算和回收策略做硬限制，而不是改回交集。

3.3 语义分支为什么选稀疏检索而不是 Encoder

表 2 的结果有点反直觉，但符合“控制面必须极轻”的约束：

  • Retrieval（TF-IDF + Top-N）：Micro-F1 0.970，延迟 2.1 ms
  • Encoder（MiniLM）：命中率也高，但 Precision 掉到 0.634，延迟 7.8 ms
  • FastText：0.17 ms，但 Micro-F1 只有 0.124，基本不可用

生产路径因此选稀疏检索。意图路由跑在 Token 生成的旁边，预算是微秒到毫秒，不是“再挂一个小模型也无所谓”。Encoder 的 Precision 崩掉，说明把多轮轨迹压成稠密向量后，工具之间的边界被抹平了——这对“选哪一个沙盒”这种需要硬决策的任务并不友好。

4. 机制二：随机预取——把准备窗口从“当前解码”延伸到“当前执行”

步内预热再好，重叠窗口也只剩当前步还没生成完的 Token。若沙盒冷启动（2–20 秒）长于这段边际生成时间，步内预热就会打不满。论文的第二个观察是：智能体会话有时间局部性——检索之后常读文档，分析之后常出图——尽管不是确定性 DAG。

SpecBox 用一阶马尔可夫过程在**沙盒依赖图（SDG）**上建模步到步转移：

  1. 统计 vi → vj 的转移计数，拉普拉斯平滑避免稀疏边概率归零。
  2. 先丢掉冷启动很轻的后继（λ=5 秒）：预热它们收益不大。
  3. 再丢掉低概率后继（τ=0.6）：控制假阳性。
  4. 最后在预算 B=1 下只预热 top-1。

B=1 非常克制。它明确拒绝“把可能后继都预热”这条退回预留部署的路。预取发生在 Tsandbox_exec(N) 期间，也就是当前工具正在跑、下一步还没提交的时候。在线更新放在后台线程，不挡前台循环。

图 11 的消融把这项机制的价值说得很干净：

  • 第 1 轮两种变体差不多（约 512–540 ms），因为还没有历史转移。
  • 从第 2 轮起，Proactive 把等待从 540 ms 打到 138 ms，第 10 轮到 97 ms；Reactive 反而升到 583 ms，末端差 6×。
  • Reactive 的每轮冷启动可到 2.90；Proactive 压在 0.24–0.83。

也就是说：跨步预取不是锦上添花，它是多轮会话里延迟不会随轮次爬升的原因。 步内预热解决“这一步猜得够不够早”，跨步预取解决“下一步的启动是不是根本不该再出现在关键路径上”。

一阶马尔可夫的边界也很清楚。开放式长时程智能体（深度研究、多文件改代码、子智能体分派）的工具依赖常常不是“只看上一个沙盒”。论文自己的退路是：预取 miss 时回退到步内意图预热，最坏延迟仍被框在当前解码阶段。这在系统上是健全的，但意味着 SpecBox 对“规划突然转向”的工作负载，收益会明显收窄到机制一。

5. 机制三：复用感知数据面——预热之后，关键路径上还剩什么

预热只能拿掉 Tenv_prep。多轮智能体还会反复查同一份文档、用不同措辞问同一件事，并把文件、图像、结构化结果经 RPC 塞回控制面。于是还剩 Tsandbox_exec 和 Tdata_io。

5.1 语义缓存：精确匹配的保守扩展

查找顺序是硬约束，不是“语义相似就复用”：

  1. 工具身份必须相同（接口兼容）。
  2. 规范化参数后做语义相似，阈值 τc（实现里写 0.8，表 4 消融用 0.6，两处不一致）。
  3. 只缓存确定性调用；命中则跳过初始化与执行。

表 4 显示收益是存在的，但幅度比前两机制克制：相对无缓存 2.91×（412 ms → 142 ms），命中率 37.4%；精确匹配已经能到 33.6% 命中、100% 旁路。语义缓存多捞到的是表层措辞变化，旁路率反而降到 84.8%——部分语义命中仍要回退执行。这符合“正确性优先于激进复用”的设计，也说明 在他们构造的轨迹里，重复计算并没有想象中那么多，真正的大头仍是冷启动掩盖。

实践上最大的风险不在相似度算法，而在“何谓确定性”。网页检索、股票、邮箱、代码仓库状态都会随时间变。若把这类工具放进语义缓存，智能体会看到过期世界。论文把非确定性调用排除在缓存外，但工程上如何自动标注，文中几乎没有展开。

5.2 带外传输：控制面继续走 MCP，载荷走共享内存

即便沙盒已热，传统 JSON-RPC 仍把日志、文件、图像序列化进请求–响应。1 GB 载荷下 JSON-RPC 约 1873 ms，带外共享内存约 6 ms（约 314×），曲线接近 O(1)。做法是控制面只带 64-bit token_id，产物写进主机 mmap 区域，智能体引擎直接读。

这一点的架构含义比加速数字更重要：

  • 兼容性：不替换 MCP/RPC 控制协议，现有工具可以不改。
  • 约束性：mmap 要求引擎与沙盒共置同一主机。这和“沙盒解耦成微服务、跨节点调度”的云原生方向是反着的。论文把跨节点留给未来的 RDMA。
  • 安全边界：共享内存桥只在受信任主机或受管集群内；沙盒逃逸、恶意智能体、基础设施被攻破不在威胁模型里。对要跑不可信代码的多租户平台，这个假设需要单独评估。

6. 实验数字应该怎么读

实现落在 AgentScope + Docker + Python，推理走阿里云百炼 Qwen3.5-Max 的生产 API。基准是从 MCPBench 的 32 个开源 MCP 工具出发、由 LLM 规划器合成的 200 条 1–10 步轨迹，不是生产集群轨迹。对比的是永久预热的 Reserved 和每次调用再实例化的 On-demand。

把端到端数字放在这组前提下来看：

延迟。 QPS=20 时 P99 为 88.7 s，相对 On-demand 的 257.2 s 约 2.9×；累积沙盒供给延迟降 4.53×，相对 Reserved 只差 10.6%。论文自己问的那句话——能否以接近 serverless 的成本拿到接近 serverful 的性能——在延迟维度上，答案基本是“可以逼近”。

内存。 峰值 49.4 GiB，相对 Reserved 的 80.6 GiB 降 45.9%；但 On-demand 峰值只有 40.4 GiB。SpecBox 并不是三者里最省内存的，它是 靠近 On-demand 的内存曲线，靠近 Reserved 的延迟曲线。多出来的那一截，就是预热/预取的投机库存。

CPU。 这是最值得注意的一点：峰值约 12.2 核，两个基线都在 15.8–15.9 核，SpecBox 比两边都低。合理原因有两条：高 QPS 下 On-demand 会把大量冷启动挤到同一时刻；Reserved 则维持过多常驻容器。缓存跳过的执行，以及错开的预热，反而降低了主机峰值争用。

读实验时需要打折的地方：

  1. 轨迹是合成的。 规划器按模板生成多轮会话，工具转移统计会比真实用户轨迹更规整，对一阶马尔可夫更友好。开放式编码智能体、并行工具调用、子智能体，可能没有这么干净的 SDG。
  2. LLM 在云端。 Tgeneration 被网络与远端排队拉长，重叠窗口更大，预热更容易藏住。本地小模型、高吞吐引擎上，步内预热的窗口会显著变窄，跨步预取的相对重要性会上升。
  3. 单机 Docker，不是 MicroVM 集群。 论文自己也说瓶颈常在应用级握手而非 OS 启动。FaaSnap / Firecracker 若把物理启动压到毫秒，SpecBox 仍有价值——因为它掩盖的是 MCP 发现、环境附着这一层——但端到端 2.9× 未必能原样复现。
  4. 结论里的 97.9% 预热命中率 在正文实验表中没有单独展开，更像汇总口径，不宜单独当核心指标引用。
  5. 正文里系统名在 SpecBox 与 Laplace 之间来回，表 4 阈值与实现阈值也不一致。不影响主结论，但说明文本仍有改名与整理痕迹。

7. 正确性模型：推测的是准备，不是执行

这一点必须单独写，否则很容易把 SpecBox 理解成“提前跑工具”。

论文的安全/正确性边界其实很克制：

  • 预测性准备只让环境就绪，在智能体提交调用之前不执行有副作用的工作；没用上的预热丢弃。
  • 缓存只覆盖确定性、接口兼容的请求。
  • 用户可见语义与 MCP 控制协议保持不变。

所以它推测的是 哪一个沙盒即将被需要，不是 工具调用的结果。这与 LLM 推测解码（先生成、再验证/丢弃 Token）不同，也与“投机执行工具并回滚”不同。设计因此能避开大多数副作用噩梦，代价是：一旦猜错，浪费的是容器资源，救不回已经走错的 Agent 轨迹——轨迹对错仍完全由上层引擎负责。

8. 和相邻工作怎么对齐

可以把它看成三块拼图之间的中间件，而不是其中任何一块的替代：

  • 对轻量沙盒运行时（Firecracker、RunD、Wasm、FaaSnap）：正交。 那些工作压缩物理启动；SpecBox 掩盖剩余就绪延迟。两者叠加才是完整故事：启动本身变短，再被重叠藏掉。
  • 对经典 serverless 预热（Mitosis、IceBreaker）：假设被打破。 那些系统默认请求独立同分布或可被时间序列预测。智能体工具调用由上下文轨迹非线性决定，必须引入 SDG 这类显式行为模型。
  • 对 vLLM / SGLang：各管一段。 它们优化 GPU 内核算与 KV 缓存，看不见主机侧沙盒摩擦。
  • 对 AgentScope / AutoGen / LangGraph：补的是它们缺的系统视角。 这些框架把工具调用当成普通函数/RPC，没有把“环境生命周期”当成一等调度对象。
  • 对 Aries 那类测量论文：诊断与处方。 Aries 说明智能体服务不能只看 Token 指标、沙盒呈空闲但突发；SpecBox 给出一种不靠永久预留、也不靠频繁快照的运行时处方。

论文还提到可迁到 Agentic RL（VERL、Slime）的 rollout 循环。这个延伸是成立的：训练时环境交互更密、重复性更强，预热/预取的 ROI 可能比在线服务更高。但这需要另一套评测，本文没有提供数据。

9. 对工程实践真正可搬走的东西

不必完整复现 SpecBox，下面几条已经能指导现有智能体平台：

  1. 不要等 tool_call JSON 闭合再拉环境。 流式输出里一旦出现稳定的工具身份信号，就可以开始镜像/容器/会话握手。即使只有关键词规则，γ 取 2 左右通常比“等完全解析”更划算。
  2. 给工具转移建图，而不是把每次调用当独立事件。 哪怕只做 top-1 后继预热，多轮会话的尾延迟也会明显不同于逐步冷启动。
  3. 控制面与数据面必须拆开。 MCP/JSON-RPC 适合指令、错误、元数据；文件、截图、向量、长日志应该走 shm / 本地盘 / RDMA 引用。否则沙盒“已经热了”，智能体仍会卡在序列化上。
  4. 投机只准备、不执行。 这是能在生产落地的前提。提前跑带副作用的工具，正确性模型会立刻崩溃。
  5. 假阳性要用预算硬封顶。 SpecBox 用 B=1、λ、τ 把预热集合锁死。只提高召回、不设预算，系统会在高峰期自己预热成 Reserved。
  6. 先打冷启动，再打缓存。 从消融看，意图预热和跨步预取是 P99 的主因；语义缓存是锦上添花。资源有限时，不要先上复杂的语义复用。

10. 局限，以及读完之后还该问的问题

SpecBox 把“智能体运行时可以是预测性的”论证到了一个可实现原型的程度，但它留下的缺口和它解决的问题一样具体：

  • 工作流形态在变。 当代智能体大量出现并行工具调用、计算机使用、子智能体、长程代码修改。ReAct 逐步单工具的 SDG 模型会不够用。并行调用会把 B=1 直接打穿，需要的是集合预取而非单后继预取。
  • 共置假设与解耦沙盒是矛盾的。 带外 mmap 把性能钉在单机拓扑上。若生产环境把 MCP 沙盒调度到别的节点或别的集群，第 3 项优化基本失效，第 1、2 项仍在。
  • 意图来自 Token 流，也就受生成风格绑架。 模型如果延迟给出工具名、先长篇思考再调用，步内预热窗口会被思考 Token 填满——这反而有利；若模型很早就以内部 ID 或极短标签点名工具，关键词画像要重做。路由特征是工具生态的函数，不是与模型无关的常数。
  • 多租户公平性几乎未谈。 误预热在单机实验里只表现为一点额外内存；在共享集群里，它会挤占别人的冷启动额度。需要准入控制、租户级预算和抢占，而不仅是 γ 和 τ。
  • 缺少与“更快的沙盒底座”的联合实验。 正交性是理论上的。若对比对象已经是快照恢复毫秒级的 MicroVM，SpecBox 的绝对收益还剩多少，文中没有数。

如果只用一个判断收束：SpecBox 证明了智能体服务的关键路径可以被时间重叠改写，而且改写点在引擎与沙盒之间的运行时，而不是模型内核或容器内核。 它给出的不是通用调度理论，而是一套针对“流式意图 + 多轮工具局部性 + MCP 解耦”的可落地中间件。对正在把 MCP 工具池铺到生产环境、又不敢为每个租户预留全套沙盒的系统来说，这篇论文的主路径——早猜、少猜、只准备不执行、控制面与数据面分离——比它报告的 2.9× 本身更有复用价值。