Lora

AI Engineering Harness: 从 Vibe Coding 到工程化

Sun, 02 Aug 2026 00:00:00 GMT

背景：Vibe 之后，正式接管

在 AI 编程助手的世界里，有一个新痛点：

AI 写了 2000 行代码，能跑，但没人敢改。没有测试，没有 CI，没有文档。

这就是 Vibe 之后的问题——你用 AI 快速产出了代码，但得到的不是一个工程交付，而是一个"技术债务黑洞"。

实际场景

想象一下这样的对话：

你: "帮我做一个用户认证系统"
Claude: (30 秒后) "完成了！放在 src/auth/"
你: (打开代码) "这...能跑，但为什么没有测试？为什么没有文档？
   为什么 Session 登录用的是 localStorage 而不是 HttpOnly Cookie？"

AI 能快速产出看起来能跑的代码，但它不会：

写测试
配置 CI
写文档
考虑安全性
预留扩展点

传统方案：你花几天时间手动补全这些。每次有新需求，都要重新手动审查。

AI Engineering Harness：让 AI Agent 自己完成从 PRD 到 Merge 的完整工程流程。

问题：AI 工程的四大痛点

痛点	影响	数据
🚧 没有测试	改了怕炸	85/108 测试通过，但还是挂了
🔴 CI 全绿，但假的	误导性强	生产环境发现严重 bug
📝 没有文档	知识碎片化	下次接手的开发者要重读所有代码
🔐 没有审查	安全隐患	AI 写入了硬编码密钥

传统方案的代价

手动补全一个 Vibe Coding 项目：

Day 1: 补测试 → 写 50 个测试用例
Day 2: 配置 CI → GitHub Actions + 3 个 workflow
Day 3: 写文档 → API 文档 + 部署指南
Day 4: 安全审查 → 手动找硬编码密钥
Day 5: 代码审查 → 逐行检查逻辑
→ 5 天，$2000+ 工程师成本

AI Engineering Harness：

输入: PRD.md
输出: 完整工程交付（测试 + CI + 文档 + 审查 + 证据包）
→ 30 分钟，$0.50 token 成本

解决方案：AI Engineering Harness

把失控的 Vibe Coding 作品，变成可验证、可维护的工程交付。

核心理念

AI Engineering Harness 不是一条 Prompt，而是一套软件工程组织操作系统。你给它一个失控的仓库，它代你组建一个由 18 类 Agent 组成的工程团队，走完整闭环：

Idea → PRD → Issue → Agent 认领 → Worktree → 实施计划
     → 实现 → 自测 → Draft PR → CI → 对抗式审查 → 修 → 再审
     → 证据闸门 → 人工审批 → 合并 → 阶段总结 → 记忆沉淀 → 下一轮

关键承诺：代码只有在 CI Pass + 至少 2 名冷启动审查员 Approved + 证据完整 时才进入 main。没有"看起来跑通了"这种状态——只有**"可验证地跑通了"**。

四个关键特性

1️⃣ 证据驱动的工程纪律

传统开发：

PR 合进去 → Done ✅

Harness：

PR → CI 绿 → 2+ 审查员 Approved → 证据齐全 → Done ✅

2️⃣ 18 类专业化 Agent

Coordinator
├── Frontend Agent
├── Backend Agent
├── Database Agent
├── Bug Hunter
├── Behavior Reviewer
├── Architecture Reviewer
├── Security Reviewer
├── UI Reviewer
├── QA Agent
├── Evidence Aggregator
└── ... (共 18 类)

3️⃣ 工作流隔离（Worktree）

每个 Issue 运行在独立的 Git Worktree 中：

主仓库: main
  ├── worktree/feature-17-auth → Issue #17 (Backend Agent)
  ├── worktree/feature-18-ui   → Issue #18 (Frontend Agent)
  └── worktree/feature-19-db   → Issue #19 (Database Agent)

好处：

✅ 并行开发，互不干扰
✅ 冲突自动检测，Conflict Resolver 处理
✅ 每个分支独立 CI 和审查

4️⃣ 上下文分层加载（L0-L3）

不让 Agent 读整个 docs/ 目录，而是分层加载：

L0: AGENTS.md + ENGINEERING.md 摘要（始终加载）
L1: 当前 Issue + 模块架构 + ADR（任务级）
L2: 相邻模块 + 最近阶段总结（按需）
L3: PDF/图片/长报告（显式需要时才加载）

核心架构：18 类 Agent 的协作

架构总览

graph TD
    User[👤 User] -->|PRD| Coordinator[🧠 Coordinator]
    Coordinator -->|分派| Frontend[🎨 Frontend Agent]
    Coordinator -->|分派| Backend[⚙️ Backend Agent]
    Coordinator -->|分派| Database[🗄️ Database Agent]

    Frontend -->|代码| Worktree1[Worktree #17]
    Backend -->|代码| Worktree2[Worktree #18]
    Database -->|代码| Worktree3[Worktree #19]

    Worktree1 -->|PR| CI[🔴 CI Gate]
    Worktree2 -->|PR| CI
    Worktree3 -->|PR| CI

    CI -->|通过| Review[🔍 Cold-Start Review]
    Review -->|Bug Hunter| Evidence[✅ Evidence Gate]
    Review -->|Behavior Reviewer| Evidence
    Review -->|Arch Reviewer| Evidence

    Evidence -->|全绿| Merge[🔀 Merge to main]
    Merge -->|阶段总结| Memory[💾 Memory Evolution]
    Memory -->|沉淀| Next[🔄 Next Round]

Agent 角色详解

🧠 Coordinator（协调者）

职责：

读 PRD，拆解成 Issue
分派 Agent 到 Worktree
维护 PROJECT_STATUS.md
人工审批闸门（鉴权/DB schema/生产密钥）

不做什么：

❌ 不写业务代码
❌ 不做技术决策
❌ 不合并代码（除非证据闸门全绿）

🎨 Frontend Agent（前端工程师）

职责：

实现 UI 组件
响应式设计
截图 + Playwright trace
Accessibility 扫描

输出：

docs/evidence/17/
├── change-summary.md
├── verification.md
├── screenshots/
│   ├── desktop.png
│   ├── tablet.png
│   ├── mobile.png
│   ├── empty-state.png
│   ├── error-state.png
│   └── loading-state.png
├── playwright-trace.zip
└── review-frontend.md

⚙️ Backend Agent（后端工程师）

职责：

API 实现
业务逻辑
异常处理
鉴权逻辑

输出：

docs/evidence/17/
├── api-trace.json
├── error-coverage.md
├── auth-negative-cases.md
└── performance-baseline.json

🐛 Bug Hunter（缺陷猎人）

职责：

冷启动审查（不读实现者的聊天历史）
只读 Issue + Plan + PR Diff + Evidence
找边界条件、竞态条件、类型错误

审查清单：

👁️ Behavior Reviewer（行为审查员）

职责：

功能是否符合 AC（Acceptance Criteria）
用户体验是否合理
边缘场景是否覆盖

闭环流程：从 Issue 到 Merge

完整流程图

9 个 Phase 详解

Phase 0: PRD → Coordinator

输入：PRD.md

Coordinator 做：

读取 PRD，提取功能点
生成 docs/product/、docs/architecture/、docs/design/
创建 memory/ 目录
生成 PROJECT_STATUS.md
创建 AGENTS.md、CLAUDE.md、ENGINEERING.md 等
生成首批 5-10 个 Issue

输出：

docs/
├── product/
│   ├── vision.md
│   └── roadmap.md
├── architecture/
│   └── system-design.md
├── design/
│   └── ui-guidelines.md
├── decisions/
│   └── ADR-001.md
memory/
├── project-memory.md
└── decisions.md
PROJECT_STATUS.md
AGENTS.md

Phase 1: Planning → Coordinator

前提：Issue 字段齐全

必须字段：

Context / Goal / Scope / Non-Goal / Related Docs / Implementation Plan /
Acceptance Criteria / Evidence Requirements / Reviewer Requirements /
Owner / Estimate

Coordinator 检查：

❌ 缺字段 → 停在 Planning，不启动
✅ 字段齐全 → 进入 Worktree Phase

Phase 2: Worktree → Owner Agent

操作：

# 为 Issue #17 创建独立 worktree
git worktree add ../myproject-worktree-17 -b feature/17-user-auth main

cd ../myproject-worktree-17

# Owner Agent 在这里工作
# Frontend Agent 实现 UI
# Backend Agent 实现 API
# Database Agent 实现 schema

每个 Worktree：

✅ 独立分支
✅ 独立 CI
✅ 独立审查
✅ 独立 Evidence 目录

Phase 3: Implementation → Owner Agent

Owner Agent 职责：

读 Plan：理解要做什么
实现代码：按 Implementation Plan 写代码
自测：跑测试，确保通过
写 Evidence：开始填充 docs/evidence/<id>/

Owner Agent 不做什么：

❌ 不合并代码
❌ 不做架构决策
❌ 不绕过审查

Phase 4: CI Gate → Coordinator

触发：Owner 推第一个 commit

Coordinator 监控：

# .github/workflows/test.yml
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm install
      - run: npm test
      - run: npm run lint

红灯处理：

🔴 CI 失败 → 停在 Phase 4
🔧 派 Recovery Agent 修复
🟢 CI 通过 → 进入 Phase 5

红线：同一类失败 ≥ 2 次 → 创建 ci tagged Issue + memory/lessons.md 一条

Phase 5: Review → Bug Hunter + Behavior Reviewer

Bug Hunter（冷启动）：

只读 Issue + Plan + PR Diff + Evidence
不读实现者的聊天或解释
输出：docs/evidence/<id>/review-bug-hunter.md

Behavior Reviewer（冷启动）：

只读 Issue + Plan + PR Diff + Evidence
检查 AC 是否满足
输出：docs/evidence/<id>/review-behavior.md

审查员规则：

✅ 必须冷启动（零上下文）
✅ 必须 Approved / Changes Requested
❌ 不能"看起来可以"
❌ 不能绕过任何 AC

Phase 6: Evidence → Evidence Aggregator

Evidence Aggregator 检查清单：

# docs/evidence/17/verification.md

## Acceptance Criteria

| AC                       | 状态       | 证据                      |
| ------------------------ | ---------- | ------------------------- |
| AC-1: 用户可以用邮箱注册 | ✅ PASS    | screenshot:register.png   |
| AC-2: 登录态持久化       | ✅ PASS    | screenshot:persistent.png |
| AC-3: 忘记密码功能       | ❌ FAIL    | 未实现                    |
| AC-4: 邮箱验证           | ⏸️ PENDING | -                         |

## Review Checklist

- [x] review-bug-hunter.md ✅
- [x] review-behavior.md ✅
- [x] fix-tasks.md ✅ (0 unresolved)
- [x] CI 绿 ✅
- [ ] AC-3 必须完成 ❌

## Verdict

❌ **NOT READY** — AC-3 未完成

Phase 7: Merge → Coordinator

合并条件：

✅ CI 绿
✅ ≥ 2 名审查员 Approved
✅ Evidence 齐全（docs/evidence/<id>/）
✅ Aggregator ✅

Coordinator 检查：

# 自动检查
- [ ] docs/evidence/17/change-summary.md
- [ ] docs/evidence/17/verification.md
- [ ] docs/evidence/17/screenshots/ (6 张)
- [ ] docs/evidence/17/review-bug-hunter.md ✅
- [ ] docs/evidence/17/review-behavior.md ✅
- [ ] docs/evidence/17/fix-tasks.md ✅
- [ ] CI 绿

全绿 → 自动合并到 main

Phase 8: Memory → Coordinator

Phase Summary：

workflows/06-phase-summary.md
→ 沉淀到 memory/phase-17-summary.md

Memory Evolution：

workflows/08-memory-evolution.md
→ 更新 memory/project-memory.md
→ 更新 memory/decisions.md

下一轮 Agent 读取这些再开工

关键技术：证据闸门

什么是证据闸门？

传统开发：

PR 合进去 → Done ✅
"我觉得可以" → 合并 ✅
"本地跑通了" → 合并 ✅

证据闸门：

PR → CI 绿 + 2+ 审查员 Approved + 证据齐全 → Done ✅

证据目录结构

docs/evidence/17/
├── change-summary.md          # 改了什么
├── verification.md            # AC 的 PASS/FAIL
├── screenshots/               # 6 态截图
│   ├── desktop.png
│   ├── tablet.png
│   ├── mobile.png
│   ├── empty-state.png
│   ├── error-state.png
│   └── loading-state.png
├── playwright-trace.zip       # 交互 trace
├── console-clean.md           # Console 无错误
├── a11y-report.md             # Accessibility 扫描
├── api-trace.json             # API 调用记录
├── error-coverage.md          # 异常覆盖
├── auth-negative-cases.md     # 鉴权负面用例
├── performance-baseline.json  # 性能基线
├── db-migration.sql           # 数据库迁移
├── db-rollback.sql            # 回滚脚本
├── db-stats.json              # Pre/Post stats
├── db-sample-rows.json        # 样本数据
├── review-bug-hunter.md       # Bug Hunter 审查
├── review-behavior.md         # Behavior Reviewer 审查
└── fix-tasks.md               # Aggregator ✅

证据检查清单

前端：

✅ 6 态截图（桌面/平板/手机/空/错/加载）
✅ Playwright trace
✅ Console 干净
✅ a11y 扫描通过

后端：

✅ API trace
✅ 异常覆盖
✅ 鉴权负面用例
✅ 性能基线

数据库：

✅ Migration + Rollback
✅ Pre/Post stats
✅ Sample rows

审查：

✅ review-bug-hunter.md
✅ review-behavior.md
✅ fix-tasks.md Aggregator ✅

CI：

✅ 绿
❌ 无 Critical/High 阻断

实战：接管一个失控的仓库

场景

你刚用 AI 写了一个项目，能跑，但：

❌ 没有测试
❌ 没有 CI
❌ 没有文档
❌ 有硬编码密钥

传统方案：花 5 天手动补全。

AI Engineering Harness：30 分钟接管。

Step 1: 安装（5 分钟）

# 一行安装
npx -y skills add lora-sys/ai-engineering-harness -g --all --full-depth

装了什么：

✅ ai-engineering-harness skill
✅ build-agent-app skill
✅ frontend-creative skill
✅ dashboard skill

支持 40 个 CLI Agent：Claude Code、Codex、Grok、Cursor、Gemini、Qwen 等。

Step 2: Quick Scan（2 分钟）

# 干跑：只打印草稿
bash skills/dashboard/scripts/scan-to-issues.sh

# 真的建 Issue
bash skills/dashboard/scripts/scan-to-issues.sh --create

Quick Scan 检测：

✅ 硬编码密钥 (HIGH) → Issue #1
✅ 缺失错误处理 (MEDIUM) → Issue #2
✅ 重复逻辑 (MEDIUM) → Issue #3
✅ 风格漂移 (LOW) → Issue #4
✅ 缺少测试 (MEDIUM) → Issue #5
⚠️ TODO 无 Issue 链接 (LOW) → Issue #6

Step 3: 接管（10 分钟）

# 在你的仓库里说
Use $ai-engineering-harness to take over this repo. Inventory the gap
between current state and harness layout; file Issues for the missing
pieces; do not edit code yet.

Coordinator 做：

盘点差距（"现状 → 期望"的 Gap）
生成 Issue（按类别归列）
不先动业务代码

输出：

📊 盘点完成
- ✅ 发现 23 个问题
- ✅ 创建 15 个 Issue
- ✅ 优先级排序

前 3 件事：
1. 硬编码密钥 → 移到环境变量
2. 缺少测试 → Quick Scan 检测到 5 个模块无测试
3. 没有 CI → 配置 GitHub Actions

Step 4: 推一个 Issue 到 Done（10 分钟）

# 推 Issue #1 到 Done
Use $ai-engineering-harness to take Issue #1 from Planning to Done.

完整闭环：

1. 写 Plan → Issue #1 的 Implementation Plan
2. Worktree → git worktree add ../proj-issue-1
3. 分派 → Backend Agent 修密钥
4. 实现 → 移到环境变量
5. 自测 → 跑测试，通过
6. Draft PR → 自动创建 PR
7. CI → 通过 ✅
8. Bug Hunter → 冷启动审查，Approved
9. Behavior Reviewer → 冷启动审查，Approved
10. Evidence → 证据齐全 ✅
11. Merge → Coordinator 自动合并
12. Memory → 阶段总结 + 记忆沉淀

Step 5: 验证（3 分钟）

# 检查证据
ls docs/evidence/1/
# ✅ change-summary.md
# ✅ verification.md
# ✅ screenshots/ (6 张)
# ✅ review-bug-hunter.md
# ✅ review-behavior.md
# ✅ fix-tasks.md

# 检查 CI
gh pr view 1 --json state,statusCheckRollup
# ✅ state: MERGED
# ✅ statusCheckRollup: SUCCESS

安装与配置

一行安装

npx -y skills add lora-sys/ai-engineering-harness -g --all --full-depth

-g：全局安装（写入用户级 skill 目录）
--all：安装到所有受支持的 CLI Agent
--full-depth：发现并安装所有 skill

精确安装

# 只装这一个 skill
npx -y skills add lora-sys/ai-engineering-harness -g -s ai-engineering-harness

# 只装到指定 agent
npx -y skills add lora-sys/ai-engineering-harness -g -a claude-code codex grok

兼容 40 个 CLI Agent

Agent	安装路径
Claude Code	`~/.claude/skills/`
Codex	`~/.codex/skills/`
Cursor	`~/.cursor/skills/`
Gemini CLI	`~/.gemini/skills/`
Qwen	`~/.qwen/skills/`
Grok	`~/.grok/skills/`
OpenCode	`~/.config/opencode/skills/`
...	共 40 个

进阶用法

跨 CLI 接力

# Claude 工作到一半，切到 Grok
Use $ai-engineering-harness. I'm continuing from another agent.
Read memory/project-memory.md and sessions/<last-id>/summary.md, then continue.

Harness 的状态全部落盘，聊天历史不会丢。

并行开发

# 分派 3 个 Issue 并行开发
Use $ai-engineering-harness to spawn parallel Owners
for Issue #20 (frontend), #21 (backend), #22 (database).

Coordinator 做：

拉 3 个 Worktree
分派对应 Agent
并行推到 PR
冲突时 Conflict Resolver 处理

CI 自愈

# CI 红了，让它自愈
CI is red on PR #23. Use $ai-engineering-harness to recover.

流程：

60 秒分类（flaky / 真缺陷 / lint / 集成 / infra）
派 Owner Agent 修复
重新跑 CI
重新走 Reviewer

30 秒拉起新项目

# 创建项目
mkdir my-saas && cd my-saas
git init
echo "# My SaaS" > README.md
git add . && git commit -m "feat: init"

# 进入任意 CLI
# Use $ai-engineering-harness to bootstrap this repo from PRD.md

Coordinator 生成：

目录骨架
首轮 Issue
ADR 模板
CI 工作流占位
PROJECT_STATUS.md → "Phase 0 / Bootstrap — Done"

效果展示

从「看起来能跑」到「可验证地跑通」。

数据对比

案例	Before → After	类型
内部工具项目（0 测试 → 47 测试）	Chaos 35 → 87	示意
install-session-hook（自审）	0 → 完整证据包	真实
Dashboard 一键接管	30 秒发现 23 个问题	示意
测试通过 ≠ 测试有效（issue #9）	9 检测器 → 10 检测器	真实
绿色的 CI 骗了我们（issue #13）	85/108 → 108/108	真实

真实案例：install-session-hook

commit: 4f311e2 → f5b26d1

问题：--status 第一版有 bug，在空环境跑会把 settings.json 创建出来。

Harness 的审查发现：

Bug Hunter 审查：7 个手动测试抓到的
修复：删了文件创建那行
Evidence：docs/evidence/15/ 完整证据包

自审诚实说：

Adversarial review 只做了一行自问自答
真生产里得 spawn bug-hunter + behavior-reviewer
没真的开 GitHub Issue #15

完整自审：docs/evidence/15/self-review.md

9 条运行原则

#	原则	为什么
1	信任证据，不信任"看起来好了"	Coordinator 不会因为"本地测试过了"就合并
2	冷启动审查	Reviewer 只读 Issue + Diff + Evidence，不读实现者的聊天
3	Issue 是工作单元	没有 Issue 不开工
4	Worktree 隔离	一个 Issue = 一个 Owner = 一个 Worktree = 一个分支
5	上下文按 L0-L3 加载	默认不加载 docs/ 全文
6	人工审批闸门	鉴权/DB schema/生产密钥 → Coordinator 暂停
7	记忆是项目状态	稳定结论写到 docs/ 与 memory/，对话历史不留
8	CI/CD 是阻塞闸门	Red CI → 停在 recovery 流程
9	本地优先	PR 提议的代码本地已有等价实现时，不直接合并

总结

核心价值

从 Vibe 到工程交付 — 把 AI 写的"能跑但不敢改"的代码变成"可验证的工程交付"
证据驱动 — 不是"我觉得可以"，而是 CI 绿 + 2+ 审查员 + 证据齐全
18 类专业化 Agent — Coordinator、Frontend、Backend、Database、Bug Hunter、Behavior Reviewer 等
Worktree 隔离 — 并行开发互不干扰
上下文分层 — L0-L3 加载，不让 Agent 读整个 docs/

适用场景

场景	用 Harness?
从 PRD 落地 MVP	✅ 必须
多 Issue 并行开发	✅ 必须
接手老项目、清理技术债	✅ 强烈推荐
复盘失序的 repo	✅ 强烈推荐
跨团队 / 跨 CLI 协作	✅ 推荐
改一行 typo	❌ 不要
一次性脚本	❌ 不要

快速开始

# 1. 安装
npx -y skills add lora-sys/ai-engineering-harness -g --all --full-depth

# 2. 接管
Use $ai-engineering-harness to take over this repo

# 3. Quick Scan
bash skills/dashboard/scripts/scan-to-issues.sh --create

# 4. 推 Issue 到 Done
Use $ai-engineering-harness to take Issue #1 from Planning to Done

# 5. 验证
ls docs/evidence/1/  # 证据齐全 ✅

开源与社区

GitHub: https://github.com/lora-sys/ai-engineering-harness

版本: 0.2.2

许可证: MIT

贡献欢迎：

新 Agent 角色
新 Workflow
新检测器
文档改进
Bug 报告

让每一行代码，都有证据。 🎯

AI Engineering Harness — Multi-agent engineering org harness for vibe-coded repos.

Free Vision Skill: 给文本模型装上一双按需调用的眼睛

Sat, 01 Aug 2026 00:00:00 GMT

背景：为什么文本模型需要"眼睛"？

在 AI 编程助手的世界里，有一个明显的鸿沟：

DeepSeek-V4-Flash、Codex、Claude Code、OpenCode — 这些强大的文本模型代码能力极强，但它们无法直接读取图片。

当用户上传一张报错截图、一张 UI 设计稿、或者一张图表时，这些 Agent 只能"看"到文件路径，而看不到实际内容。

实际场景

想象一下这样的对话：

用户: "帮我修复这个错误"
(上传了一张终端报错截图)

DeepSeek: "我看到你上传了一个文件 error.png，
但我无法读取图片内容，请告诉我错误信息是什么。"

这种体验非常糟糕。用户期望 Agent 能直接理解图片内容，而不是手动描述。

问题：传统方案的四大痛点

常见的解决方案是把图片交给视觉模型，生成一段详细描述，再塞回主模型：

图片 → 视觉模型 → 长描述 → 主模型

但这种方法会带来四大问题：

问题	影响	数据
💸 Token 消耗高	每次调用 2000-5000 tokens	成本高
🗑️ 无关描述多	视觉模型输出大量不需要的内容	上下文污染
🧹 上下文污染	主模型上下文被长描述占满	性能下降
🧠 越权推理	视觉模型替主模型做决策	责任不清

举个实际例子

当用户上传一张 400x300 的错误截图时：

传统方案:

视觉模型输出: "这张图片显示了一个终端窗口，其中包含一个 Python
错误信息。错误类型是 ModuleNotFoundError，具体信息是无法找到
名为 'requests' 的模块。错误发生在文件 /Users/user/project/app.py
的第 42 行..."
→ 约 3000 tokens

Free Vision Skill:

VEP/1|src=zhipu/glm-4.6v-flash|m=error|a="ModuleNotFoundError: No module named 'requests'"|t="app.py:42"|c=0.97
→ 约 130 tokens

节省: 95%+ token 消耗 🎯

解决方案：Free Vision Skill

Free Vision Skill 是一个低 Token 消耗的视觉证据编译器，专门为文本-only 的 AI Agent 设计。

核心理念

视觉模型只负责"看见"，主模型继续负责"思考"。

工作流程

图片 + 问题
  ↓
免费视觉 API（只提取当前任务需要的事实）
  ↓
压缩为 VEP（Visual Evidence Packet）
  ↓
DeepSeek / Codex / Claude Code / OpenCode 继续推理

关键特性

特性	说明
🎯 低 Token 消耗	50-150 tokens，比完整描述节省 90-95%
🔄 自动降级	Provider 限流时自动切换到备用服务
💾 智能缓存	TTL + LRU 策略，命中率可达 90%+
🔐 安全存储	macOS Keychain、Linux Secret Service、Windows Credential Manager
🌍 13 个 Provider	国内 4 个 + 全球 9 个，全面覆盖
⚡ 性能优化	32.5x 加速（健康检查从 65s → 2s）
🔌 VEP/1 协议	极简视觉证据包格式

核心技术：VEP/1 协议

VEP = Visual Evidence Packet（视觉证据包）

视觉模型不返回完整分析，只返回事实：

VEP 格式

VEP/1|src=zhipu/glm-4.6v-flash|m=error|
a="Cannot find module ethers"|
t="src/app.ts:42"|
e=[dependency error]|
c=0.97

字段说明

字段	含义	示例
`VEP/1`	协议版本	VEP/1
`src`	Provider 和模型	`zhipu/glm-4.6v-flash`
`m`	任务模式	`error` / `ocr` / `ui` / `chart`
`a`	直接答案	`"Cannot find module"`
`t`	OCR 文本	`"src/app.ts:42"`
`o`	关键对象	`[button, input, modal]`
`e`	可见错误	`[overlapping, clipped]`
`v`	关键值	`["$99", "2024-12-31"]`
`c`	置信度	`0.97`
`cache`	缓存状态	`cache=hit`

Token 对比

场景	VEP 大小	主模型接收	传统方案	节省
错误提取	~150 chars	~50 tokens	2000+ tokens	97% ⬇️
UI 审计	~400 chars	~80 tokens	3000+ tokens	97% ⬇️
OCR 表格	~500 chars	~120 tokens	4000+ tokens	97% ⬇️
图表分析	~300 chars	~70 tokens	2500+ tokens	97% ⬇️

架构设计

系统架构

┌─────────────────────────────────────────────────────────────┐
│                     CLI Layer (free-vision)                  │
│  see | doctor | providers | login | logout | cache          │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│                    Provider Registry                         │
│  13 providers (zhipu, modelscope, openrouter, groq, ...)    │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│                  Smart Cache (TTL + LRU)                     │
│  - SHA-256 key based on image+question+provider+model       │
│  - TTL: 24 hours, Max: 1000 entries                         │
│  - Hit rate: 90%+ achievable                                │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│              Concurrency Control (RequestPool)               │
│  - Max concurrency: 3                                       │
│  - Exponential backoff retry                                │
│  - Rate limiting (token bucket)                             │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│                    Vision API Layer                          │
│  OpenAI-compatible API endpoints                            │
└─────────────────────────────────────────────────────────────┘

核心模块

1. Provider System (`src/providers.ts`)

// 13 个 Provider 配置
export function resolveProviderOrder(requested: string, region: Region): ProviderConfig[] {
	if (requested !== 'auto') return [getProvider(requested)];

	// 自动降级：优先同区域，然后 fallback
	const preferred = registry.providers
		.filter((p) => p.region === region)
		.sort((a, b) => a.priority - b.priority);

	const fallback = registry.providers
		.filter((p) => p.region !== region)
		.sort((a, b) => a.priority - b.priority);

	return [...preferred, ...fallback];
}

2. 智能缓存 (`src/cache.ts`)

interface CacheEntry {
  value: string;
  timestamp: number;        // TTL 检查
  accessCount: number;      // LRU 优先级
  lastAccess: number;       // 最近访问
  size: number;             // 内存追踪
}

// TTL + LRU 双重策略
async get(key: string): Promise<string | null> {
  // 1. 检查 TTL
  if (Date.now() - entry.timestamp > DEFAULT_TTL_MS) {
    await rm(filePath); // 过期删除
    return null;
  }

  // 2. 更新访问信息
  entry.accessCount++;
  entry.lastAccess = Date.now();

  // 3. 检查是否需要 LRU 清理
  await this.evictIfNeeded();
}

3. 并发控制 (`src/pool.ts`)

class RequestPool<T> {
	// 指数退避重试
	private async execute(item): Promise<void> {
		for (let attempt = 0; attempt <= maxRetries; attempt++) {
			try {
				const value = await this.withTimeout(fn(), timeoutMs);
				return resolve({ success: true, value });
			} catch (error) {
				// 指数退避: 1000ms → 2000ms → 4000ms
				const delay = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
				await sleep(delay);
			}
		}
	}
}

4. VEP 生成 (`src/vep.ts`)

export function toVep(result: VisionResult, maxChars: number): string {
	const parts = [
		'VEP/1',
		`src=${result.provider}/${result.model}`,
		`m=${result.mode}`,
		result.answer ? `a="${result.answer}"` : '',
		result.text ? `t="${result.text}"` : '',
		result.issues?.length ? `e=[${result.issues.join(',')}]` : '',
		`c=${result.confidence?.toFixed(2)}`
	]
		.filter(Boolean)
		.join('|');

	return compact.slice(0, maxChars);
}

性能优化：32.5x 加速

健康检查优化

优化前（v0.3 及以前）:

// 串行检查，每个 5s timeout
for (const provider of providers) {
	await checkProviderHealth(provider); // 5s each
	await sleep(1000); // 批次延迟
}
// 13 个 provider × 5s = 65s

优化后（v0.4）:

// 并发批次检查
const batchSize = 3;
for (let i = 0; i < providers.length; i += batchSize) {
	const batch = providers.slice(i, i + batchSize);
	await Promise.all(batch.map((p) => checkProviderHealth(p)));
	await sleep(500); // 批次延迟
}
// 13 个 provider: [3 + 3 + 3 + 1] = ~2s

// 加速倍数: 32.5x 🚀

缓存性能

TTL + LRU 策略:

缓存命中率: 90%+
TTL: 24 小时
最大条目: 1000
LRU 淘汰: 最少访问优先

实测数据:

Hit Rate:     87.5% (7/8)
Misses:       1
Evictions:    0
Size:         8 entries
✅ Cache is effective (>50% hit rate)

并发控制

RequestPool 配置:

{
  maxConcurrency: 3,    // 最大并发
  timeoutMs: 30000,     // 超时时间
  maxRetries: 2,        // 最大重试
  baseDelayMs: 1000,    // 基础延迟
  maxDelayMs: 10000     // 最大延迟
}

指数退避策略:

尝试 1: 失败 → 延迟 1000ms
尝试 2: 失败 → 延迟 2000ms
尝试 3: 失败 → 放弃 (total: 3000ms)

实际使用场景

1️⃣ 错误截图分析

free-vision see --image ./error.png \
  --question "只提取错误信息和行号"

VEP 输出:

VEP/1|src=zhipu/glm-4.6v-flash|m=error|
a="Cannot find module 'lodash'"|
t="webpack.config.js:15"|
e=[module resolution error]|
c=0.98

主模型继续推理: "错误是找不到 lodash 模块，在 webpack.config.js:15。解决方案：运行 npm install lodash"

2️⃣ UI 审查

free-vision see --image ./ui-screenshot.png \
  --question "列出所有被裁切、重叠或禁用的 UI 元素"

VEP 输出:

VEP/1|src=zhipu/glm-4.6v-flash|m=ui|
o=[{name:"Submit",issue:"disabled"},{name:"Avatar",issue:"clipped"}]|
c=0.95

3️⃣ OCR 表格提取

free-vision see --image ./table.png \
  --question "提取所有文本和表格结构"

VEP 输出:

VEP/1|src=zhipu/glm-4.6v-flash|m=ocr|
a="Q3 销售报表"|
t=["产品","销售额","增长率"],["A",12000,"15%"],["B",8500,"8%"]|
c=0.92

4️⃣ 图表数据提取

free-vision see --image ./chart.png \
  --question "只返回图表标题、趋势和三个关键值"

VEP 输出:

VEP/1|src=zhipu/glm-4.6v-flash|m=chart|
a="月度营收增长"|
v=[45200,58300,72100]|
c=0.96

5️⃣ 自动裁剪优化

free-vision see --image ./screenshot.png --auto-crop \
  --question "只提取错误信息"

裁剪结果:

✂️  Cropped: 400x300 → 369x58
   Reduction: 82%
   Saved to: ./screenshot.cropped.png

效果: 图片大小减少 82%，进一步降低 token 消耗。

支持的 AI Agent

Free Vision Skill 不绑定某个主模型，适合所有文本-only 的 Agent：

🤖 Claude Code

# Claude Code Hook 自动检测图片
npx skills add lora-sys/free-vision-skill

🤖 Codex

# 一键安装脚本
curl -fsSL https://raw.githubusercontent.com/lora-sys/free-vision-skill/main/installers/codex-install.sh | bash

🤖 OpenCode

{
	"agents": {
		"coder": {
			"skills": ["free-vision"],
			"vision": {
				"provider": "auto",
				"region": "cn",
				"auto-detect-images": true
			}
		}
	}
}

🤖 DeepSeek / 其他

# 通用调用
free-vision see --image ./screenshot.png --question "你的问题"

开发历程与未来规划

v0.1.0 — MVP（已完成 ✅）

v0.2 — 集成增强版（已完成 ✅）

Claude Code Hook 智能识别
Provider 健康检查
Codex 一键安装脚本
OpenCode Agent 集成

v0.3 — 高级功能（已完成 ✅）

Windows Credential Manager
VEP Schema Validator
Image auto-crop（--auto-crop）

v0.4 — 性能优化（已完成 ✅）

Cache TTL + LRU eviction
Request pool with configurable concurrency
Rate limiter（token bucket）
Exponential backoff retry
Parallel failover for providers
Performance tests（30/30 passing）

v1.0 — 生产就绪（规划中）

总结

Free Vision Skill 解决了一个真实且迫切的问题：如何让文本-only 的 AI Agent 低成本地理解图片。

核心价值

90-95% Token 节省 — 从 2000-5000 tokens 降至 50-150 tokens
智能缓存 — 90%+ 命中率，TTL + LRU 双重策略
性能优化 — 32.5x 加速（健康检查 65s → 2s）
跨平台支持 — macOS、Linux、Windows
13 个 Provider — 国内 4 + 全球 9，自动降级

适用场景

✅ 报错截图分析
✅ UI 审查和设计稿分析
✅ OCR 表格提取
✅ 图表数据提取
✅ 代码截图理解
✅ 任何需要视觉理解的文本 Agent

开源与社区

GitHub: https://github.com/lora-sys/free-vision-skill

许可证: MIT

贡献欢迎:

新 Provider Adapter
VEP 压缩改进
本地模型支持（Ollama 等）
Windows Keychain 支持
Agent 集成示例

致谢

感谢以下开源视觉模型和 API 提供商：

智谱 AI (Zhipu AI) — GLM 系列模型
阿里 ModelScope — 魔搭社区
OpenRouter — 统一 API 网关
Groq — 高速推理
Google Gemini — 多模态 AI
Mistral AI — 欧洲开源 AI
Cloudflare Workers AI — 边缘 AI
Ollama — 本地模型运行

先看见，再压缩，再推理。 👁️

Free Vision Skill — low-token visual evidence compiler for text-only coding agents.

用 MiniMax 为 Hermes Agent 接入图片与视频生成

Fri, 10 Jul 2026 00:00:00 GMT

如果你已经在用 Hermes Agent，但没有本地 GPU、也不想折腾 ComfyUI，那么这篇教程很适合你。我会把 MiniMax（海螺 / minimax）的图片和视频生成能力同时接进 Hermes，并且在一个 pip 包里发布两套后端：hermes-minimax-media，覆盖文生图、图生图、文生视频、图生视频、人物参考视频五种模式。

整个过程我用的是真实的 CN endpoint（api.minimaxi.com）和真实的 API key 跑通端到端。

为什么选 MiniMax

Hermes Agent 内置的 image_generate 工具原本只支持少数几个海外后端（FAL、xAI、OpenAI、Krea、Replicate）。MiniMax 海螺这边有两个我特别想用的能力：

image-01：单 API 同时支持文生图和图生图（subject_reference 单参考图），出图质量稳定，单张图消耗低
Hailuo 系列视频模型：异步任务模型，支持文生视频、图生视频、首尾帧视频、人物参考视频。1080P 25fps 真实可用，不是 demo 玩具

而且 MiniMax 有两个 endpoint：

区域	域名	用户中心
国内（CN）	`api.minimaxi.com`	https://api.minimaxi.com
国际（global）	`api.minimax.io`	https://www.minimax.io

插件会自动选：CN 优先（你用的是国内账户就走国内），global 作为兜底。

实现一个插件要做什么

Hermes 提供的 ImageGenProvider 和 VideoGenProvider 两个抽象类都已经把脏活干完了（环境变量加载、URL 缓存、错误格式化）。我们只需要写两个文件：

~/.hermes/plugins/image_gen/minimax/
├── __init__.py        # 业务逻辑
└── plugin.yaml        # 插件元数据

~/.hermes/plugins/video_gen/minimax/
├── __init__.py
└── plugin.yaml

plugin.yaml 极简：

name: minimax
version: 1.0.0
description: 'MiniMax image generation backend. ...'
author: NousResearch
kind: backend

image-01 后端实现要点

主要工作是组装 HTTP 请求。文档里几个非显然的点：

路径用 aspect_ratio 而不是 size（和 OpenAI、StepFun 不一样）：

{
	"model": "image-01",
	"prompt": "a tiny red apple on a white plate, studio lighting",
	"aspect_ratio": "1:1", // 不是 "1024x1024"
	"response_format": "url", // 还可以是 "base64"
	"n": 1,
	"prompt_optimizer": true
}

图生图用 subject_reference 包裹图片 URL：

"subject_reference": [
  { "type": "character", "image_file": "https://..." }
]

返回结构是嵌套的：

{
	"data": { "image_urls": ["https://..."] },
	"metadata": { "success_count": "1" },
	"base_resp": { "status_code": 0, "status_msg": "success" }
}

把请求组装好、调用 _call_api、拿到 URL 后用 Hermes 自带的 save_url_image 落盘到 ~/.hermes/cache/images/，整个 generate 流程就完了。

Hailuo 视频后端实现要点

视频生成和图片最大的区别是异步：

1. POST  /v1/video_generation          → task_id
2. GET   /v1/query/video_generation    (10s 轮询)  → file_id
3. GET   /v1/files/retrieve?file_id=…  → download_url
4. GET   <download_url>                → 字节流 → 本地缓存

超时策略：单次任务最长 5 分钟，10 秒轮询一次。Hailuo-2.3 6s 768P 实测 30-90 秒返回，10s 1080P 偶尔会 3-4 分钟。模型选择上要注意：

模型	6/10s	768P/1080P	T2V	I2V	人物参考
`MiniMax-Hailuo-2.3`	✅	✅	✅	✅	❌
`MiniMax-Hailuo-02`	✅	✅	✅	✅	❌
`S2V-01`	只 6s	只 1080P	❌	❌	✅

⚠️ 一个坑：官方文档示例里写的是 720P，但实际 API 不接受。提交 720P 会立刻报错 does not support resolution 720P, supported resolutions: 768P, 1080P。我的插件把 720P 自动 coerce 成 768P，调用方可以无感。

端到端测试效果

image-01 文生图（1:1 方形）

a tiny red apple on a white plate, studio lighting, photorealistic

单图 5 秒左右返回，~150KB。

image-01 文生图（16:9 横向）

a cute cat astronaut floating in a colorful nebula, digital art, vibrant

futuristic tokyo cityscape at night, neon lights, rain reflections, cyberpunk

Hailuo-2.3 文生视频（6s @ 768P）

a calico cat napping in a sunbeam, soft cinematic lighting, gentle breathing motion

视频演示 GIF 已从上游仓库移除；保留下方 MP4 原片链接作为可验证演示。

MP4 原片 (608KB)

86 秒返回，输出 1366×768 h264 mp4，每帧都有微小变化（呼吸感）。

本地接入步骤

1. 安装包

pip install hermes-minimax-media

2. 配 key

编辑 ~/.hermes/.env：

# CN endpoint（你目前在用的）
MINIMAX_CN_API_KEY=eyJ...

# 国际 endpoint（兜底）
MINIMAX_API_KEY=eyJ...

3. 启用插件

编辑 ~/.hermes/config.yaml：

plugins:
  enabled:
    - image_gen/minimax
    - video_gen/minimax
    # ... 其他插件

image_gen:
  provider: minimax
  minimax:
    model: image-01

video_gen:
  provider: minimax
  minimax:
    model: MiniMax-Hailuo-2.3 # 或 Hailuo-02 / S2V-01

4. 重启 Hermes

hermes gateway restart

之后直接对 Hermes 说：

"画一只戴墨镜的猫" → 走 image-01
"做一个 6 秒的城市黄昏延时摄影" → 走 Hailuo-2.3
"让这张海浪照片动起来"（+ 传 image_url） → 走 Hailuo-2.3 I2V
"让这个角色挥手"（+ 传人物照片） → 走 S2V-01

仓库里有什么

整个项目都开源在 github.com/lora-sys/hermes-minimax-media：

src/hermes_minimax_media/plugins/image_gen/minimax/ — image-01 后端
src/hermes_minimax_media/plugins/video_gen/minimax/ — Hailuo 视频后端
tests/ — 20 个 pytest 用例（不需要 API key）
.github/workflows/ci.yml — py3.11/3.12 测试 + ruff lint + tag 时自动发 PyPI
docs/assets/ — README 用的 3 张示例图 + 1 个 demo.gif

端到端验证记录

我跑过的真实结果（不靠模拟）：

验证项	结果
`image-01` T2I 调 `api.minimaxi.com`	✅ 5 秒返回 155KB jpg
`Hailuo-2.3` T2V 异步提交	✅ task_id 立即返回
10s 轮询 task status	✅ 86 秒变 Success
解析 `file_id` + `download_url`	✅ 拿到 608KB mp4
双 endpoint 自动选 CN	✅ env 设了 `MINIMAX_CN_API_KEY` 时走 `api.minimaxi.com`
`pip install -e .` 包级导入	✅ 两个 provider 都能 import
`pytest tests/`	✅ 20 passed
`ruff check src/ tests/`	✅ All checks passed
`twine upload dist/*`	✅ https://pypi.org/project/hermes-minimax-media/

和之前 StepFun 那篇的关系

我之前写过一篇用 StepFun 给 Hermes 接入图片生成，流程上是同构的（本地插件 → PyPI → CI → 博客），但这两个插件互补：

StepFun：step-image-edit-2 极快（1-2 秒），适合对话中快速出图迭代；图编辑是它的强项
MiniMax：image-01 质量稳，还多一个视频能力；如果你需要"图"和"视频"同时在一个后端里，minimax 这套更划算

而且两个插件可以同时启用，在 config.yaml 里改 image_gen.provider 就能切换，不需要卸载任何一个。

如果你用 StepFun 的话记得也看一眼那边的"避坑"：StepFun 同一个 key 有 /v1/ 和 /step_plan/v1/ 两套 endpoint，分别扣现金和扣订阅 Credit。我自己第一次发版没注意，被按张扣了两周余额才发现。StepFun 那篇的"避坑"小节和 GitHub README 里有完整诊断 + 1 段可直接粘贴给你自己 agent 的排查 prompt。

下一步计划

跑 image-01 的 i2i（subject_reference）真实样例
跑 S2V-01 人物参考视频的真实样例
适配 Hailuo-02 的 start-end frame 模式（传 first_frame_image + last_frame_image 两个 url）

如果你也想接 MiniMax、但懒得自己写包，hermes-minimax-media 直接 pip install 就能用，遇到问题在 GitHub 提 issue 即可。

用 StepFun 为 Hermes Agent 接入图片生成能力：零 GPU、零 ComfyUI、开箱即用

Fri, 10 Jul 2026 00:00:00 GMT

如果你已经在用 Hermes Agent，但又苦于没有本地 GPU、不想折腾 ComfyUI，又希望把生图能力直接变成 Hermes 可调用的一等公民，那么这篇教程很适合你。

我会以**“从零到发布”**的方式，记录一次真实可复用的接入过程：本地插件开发、配置接入、功能验证、独立仓库打包、PyPI 发布、CI 与推广。

为什么选 StepFun

本地生图常见方案是 Stable Diffusion + ComfyUI，但这通常意味着：

需要 NVIDIA 显卡或 Apple Silicon
需要下载大体积模型和依赖
需要一定显存与系统配置
维护成本高，换机器就要重新配置

StepFun 的优势恰恰相反：

云端 API：模型部署在服务端，本机只需要能发 HTTP 请求
延迟低：step-image-edit-2 在 1-2 秒内可返回结果
能力强：支持文生图、图片编辑、图生图
无需本地 GPU：只要有 API Key 就能用
插件化接入：可以直接挂进 Hermes 的 image_gen 体系

先确认你的环境

项目	要求
Hermes Agent	已安装并可运行
Python	3.11+
StepFun 账号	已注册，可访问 https://platform.stepfun.com
API Key	已创建，格式通常为 `step-...`

如果你还没有 StepFun 账号，先去平台注册并创建一个 API Key。

第一步：确认 Hermes 的插件目录结构

Hermes 的本地插件目录一般在：

~/.hermes/plugins/

图片生成插件一般放在：

~/.hermes/plugins/image_gen/

目前官方仓库中已经有一些参考实现，例如：

openai
fal
krea
xai

我们会新建一个目录，专门承载 StepFun 后端：

~/.hermes/plugins/image_gen/stepfun/

第二步：编写插件清单 `plugin.yaml`

先新建目录：

mkdir -p ~/.hermes/plugins/image_gen/stepfun

再创建 plugin.yaml：

name: stepfun
version: '1.0.0'
description: StepFun image generation backend for Hermes Agent
requires_env:
  - STEPFUN_API_KEY

这里的 requires_env 很重要，它告诉 Hermes 这个插件启动前必须读取 STEPFUN_API_KEY，避免运行时才发现缺配置。

第三步：实现插件主逻辑

新建 __init__.py。这里的关键是实现 ImageGenProvider，并向 Hermes 暴露出三件事：

支持的模型列表
生图能力声明
真正的生成逻辑

模型与能力声明

MODELS = [
    {"id": "step-image-edit-2", "name": "Step Image Edit 2", "speed": "fast"},
    {"id": "step-2x-large", "name": "Step 2X Large", "speed": "high-quality"},
    {"id": "step-1x-medium", "name": "Step 1X Medium", "speed": "balanced"},
]

CAPABILITIES = {
    "modalities": ["text", "image"],
    "max_prompt_length": 1024,
    "supports_edits": True,
    "supports_image_to_image": True,
}

生成逻辑的核心流程

def generate(self, prompt, model=None, aspect_ratio="square", steps=4, seed=None):
    model_id = model or self.default_model()

    if model_id == "step-image-edit-2":
        return self._call_text_to_image(prompt, aspect_ratio, steps, seed)

    if model_id in ("step-2x-large", "step-1x-medium"):
        return self._call_image_to_image(prompt, aspect_ratio, steps, seed)

    raise ValueError(f"Unsupported model: {model_id}")

这意味着插件会在内部根据模型自动路由请求，而不是让上层手动判断。

⚠️ 避坑：两个 endpoint，两套计费

在我做完插件正式发 v0.1.0 之后用了一阵子，发现账单"诡异"——账户余额每天被扣几毛钱，但 Step Plan 订阅页面的 Credit 用量没怎么动。后来排查发现：

StepFun 对同一个 API key 提供两套独立的 endpoint，走不同的计费通道：

路径	计费方式	适用
`https://api.stepfun.com/v1/...`	账户余额按张扣钱（现金）	开放平台用户
`https://api.stepfun.com/step_plan/v1/...`	扣 Step Plan 订阅的 Credit 额度	Step Plan 订阅用户

两个路径用同一个 key 都能通过认证，但账单走完全不同的通道。我最初插件的默认 STEPFUN_BASE_URL 写的是 /v1/...，所以 Step Plan 订阅用户（包括我自己）也在被按张扣钱。

修复（v0.1.1 修复，1 行改动）：

STEPFUN_BASE_URL = os.environ.get(
    "STEPFUN_BASE_URL",
    "https://api.stepfun.com/step_plan/v1"  # 改这里：从 /v1 改成 /step_plan/v1
)

如果你不是 Step Plan 订阅用户（用的是普通开放平台账户 + 充值），保持默认 /v1 即可。如果你是 Step Plan 订阅用户，升级到 0.1.1+ 之后所有调用会走 Credit 套餐，不再扣现金。

升级命令：

pip install --upgrade hermes-stepfun-imagegen

如果你想看完整的"症状 → 排查 → 修法"诊断流程（含可直接粘贴给你自己 agent 的 7 步排查 prompt），看 README 的 Troubleshooting → My usage is billing my cash balance, not my Step Plan credit。

第四步：调用 StepFun API

StepFun 的图片生成接口通常采用类似 OpenAI 的 /v1/images/generations 风格，也可能提供专门的 edits 与 image2image 端点。这里主要注意三点：

请求头带 Authorization: Bearer ***
参数里要传 model、prompt，有时还要带 size、n
响应里通常返回图片 URL 或 base64，需要本地落盘保存，便于 Hermes 后续读取

一个简化版调用思路如下：

import os
import requests

API_KEY = os.environ["STEPFUN_API_KEY"]
BASE_URL = os.environ.get("STEPFUN_BASE_URL", "https://api.stepfun.com/v1")

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json",
}

payload = {
    "model": "step-image-edit-2",
    "prompt": prompt,
    "size": aspect_ratio_to_size(aspect_ratio),
    "n": 1,
}

resp = requests.post(
    f"{BASE_URL}/images/generations",
    headers=headers,
    json=payload,
    timeout=60,
)
resp.raise_for_status()
data = resp.json()

拿到返回后，再根据 URL 或 base64 保存为本地图片即可。

注意：StepFun 的 size 参数格式可能和常规写法不同。根据实际 API 文档，要确认它是用 1024x1024、1024*1024 还是 height x width 顺序。

第五步：配置 Hermes 启用插件

插件代码写好后，修改 Hermes 配置：

plugins:
  enabled:
    - image_gen/stepfun

image_gen:
  provider: stepfun
  model: step-image-edit-2

再重启 Hermes：

hermes gateway restart

重启后，主模型调用 image_generate 工具就会自动走 StepFun。

第六步：本地验证三种模式

1. 文生图

提示词示例：

A cute cat astronaut floating in space, digital art

预期行为：插件调用文生图接口，返回一张图片路径。

2. 图片编辑

上传一张现有图片，再给一个编辑指令：

把这张图换成水墨画风格

预期行为：插件调用 edits 接口，返回修改后的图片。

3. 图生图

基于参考图生成新图：

把这张照片转成梵高风格

预期行为：插件调用 image2image 接口，返回风格化后的图片。

如果你在本地已经完成验证，恭喜你，核心能力已经通了。

第七步：把插件独立成仓库

这一步的意义是：让插件变成可分享、可安装、可版本发布的独立项目，而不是只活在 ~/.hermes/plugins/ 里。

标准目录结构如下：

hermes-stepfun-imagegen/
├── pyproject.toml
├── README.md
├── LICENSE
└── src/
    └── hermes_stepfun_imagegen/
        ├── __init__.py
        └── plugin.yaml

pyproject.toml 里最关键的是 entry point：

[project.entry-points."hermes_agent.plugins"]
stepfun-imggen = "hermes_stepfun_imagegen"

这样别人就能直接：

pip install hermes-stepfun-imagegen

第八步：发布到 PyPI

进入仓库目录，构建并上传：

cd hermes-stepfun-imagegen
python -m build
twine upload dist/*

上传成功后，你的插件就变成了：

pip install hermes-stepfun-imagegen

第九步：GitHub Actions CI

为了让仓库看起来专业，也为了让发布更自动化，我们加一个 GitHub Actions workflow：

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  release:
    types: [published]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.11', '3.12']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install pytest pytest-cov
      - run: pip install -e .
      - run: python -m pytest tests/ -v --tb=short
      - run: pip install ruff
      - run: ruff check src/ tests/

  build:
    runs-on: ubuntu-latest
    needs: test
    if: github.event_name == 'release'
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install build twine
      - run: python -m build
      - run: twine check dist/*
      - run: twine upload dist/*
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}

之后在 GitHub 仓库的 Settings → Secrets 里配置 PYPI_API_TOKEN，就可以实现“打 tag 就自动发布”。

第十步：推广到社区

仓库准备好了，接下来就是让更多人知道它。

GitHub 优化

给仓库加上 Topics：

hermes-agent
stepfun
image-generation
ai-plugin
stable-diffusion

更新仓库描述，让它更容易被搜索到。

Discord 社区

Nous Research Discord 的 #plugins-skills-and-skins 是官方推荐的插件分享渠道。文案思路：

Hi everyone! I've built a StepFun image generation plugin for Hermes Agent.

Features:
- Text-to-image with step-image-edit-2 (~1-2s)
- Image editing
- Image-to-image with step-2x-large / step-1x-medium
- Drop-in plugin for ~/.hermes/plugins/image_gen/stepfun/

Requires STEPFUN_API_KEY from https://platform.stepfun.com

Feedback welcome!

内容平台

如果你的博客已经上线，这篇文章本身就是最好的入口。还可以把 Demo GIF、生成截图、对比评测一起放进去，形成完整的宣传素材。

小结

这篇文章记录了一个完整链路：

Hermes Agent + StepFun API Key
    -> 本地插件开发
    -> 配置接入
    -> 功能验证
    -> 独立仓库
    -> PyPI 发布
    -> CI 自动化
    -> 社区推广

如果你只是想本地用起来，重点做好前三步就够了；如果你想分享给社区，第七步到第十步才真正决定项目能走多远。

Vercel Eve Agent Framework

Sun, 05 Jul 2026 00:00:00 GMT

Executive Summary

Vercel Eve is an open-source Agent framework launched by Vercel in June 2026, and as of 2026-07-03 it remains in Beta/Preview. Its core proposition is not "another agent loop SDK," but modeling an Agent directly as a directory in the filesystem: instructions.md handles long-term system prompts, agent.ts handles runtime configuration and model selection, and directories like tools/, skills/, channels/, connections/, subagents/, schedules/ each carry capabilities, channels, external integrations, sub-agents, and scheduled tasks; the framework compiles these files through auto-discovery into an Agent application running on Vercel Functions with production-grade features: durable execution, event streams, sandbox execution, observability, approvals, and human-in-the-loop collaboration.

From a design perspective, Eve's differentiation comes down to four advantages. First, it's a strongly opinionated filesystem-first programming model that reduces the structural complexity of multi-agent, multi-channel, multi-tool systems. Second, it builds durable execution directly on Vercel Workflows, allowing sessions/turns to recover across cold starts, redeployments, and long pauses. Third, it treats sandboxes as first-class citizens, isolating untrusted code execution by default in a separate security context. Fourth, it's deeply coupled with AI Gateway, Connect, Observability, and OIDC, making model access, credential proxying, tracing, and access control smoother on the Vercel platform.

If your team runs primarily on Vercel, wants to treat Agents as deployable backend systems rather than "model call scripts," and needs Slack/Web/GitHub/scheduled tasks/approvals/long-running/observable production capabilities, Eve will likely ship faster than generic frameworks. Conversely, if you need cross-cloud/self-hosted infrastructure portability, or your team prefers graph-based orchestration (LangGraph), OpenAI-native tools and containers (OpenAI Agents SDK), or framework-agnostic TypeScript AI application frameworks (Mastra), you should evaluate platform binding, runtime assumptions, and migration costs more carefully. This assessment comes from official documentation descriptions of runtime structure, persistence, sandboxes, channels, deployment, and platform service dependencies, as well as comparative analysis with other official framework documentation.

Regarding maturity, Eve's strengths are Vercel-backed ownership, comprehensive documentation, rich templates, and a growing GitHub community. As of this writing, official documentation explicitly labels it Beta, the GitHub repository has approximately 3.2k stars and 252 forks, and official/related templates already cover typical scenarios including Web Chat, Slack content assistants, personal agents, PR triage, and browser agents. This indicates a fairly clear "productized framework" outline, but it should still be treated as a rapidly evolving early-stage framework. Production adoption should be accompanied by approval strategies, authentication, evaluations, cost guardrails, and rollback plans.

Design Philosophy and Overall Architecture

Eve's basic abstraction is: "an agent is a directory." Both the official documentation and repository emphasize that Eve doesn't use one giant configuration object to declare a system; instead, each concern of an agent is placed in an explicit file location, and the framework convention-discovers these files, compiling them into applications that can run locally, be exposed via HTTP, connect to Slack and other channels, and persist across multiple durable turns.

This design makes an Eve project analogous to "Next.js for Web, Eve for Agents." The official release announcement explicitly uses the Next.js analogy, suggesting the Agent domain has reached the point where repetitive infrastructure can be abstracted into a framework: you no longer stitch together durable loops, auditing, streaming, approvals, sandboxes, and channel integrations yourself — you mainly define "who the agent is, what it knows, and how it interacts with the world."

Platform-wise, Eve on Vercel isn't a single-point feature but a composition of several platform capabilities. Vercel documentation explicitly lists: Vercel Functions for agent routing and tool execution, Vercel Workflows for session persistence and recovery, Vercel Sandbox for isolated execution, AI Gateway for model routing and provider fallback, Vercel Connect for external credentials and connections like OAuth/API keys, and Observability / Agent Runs for runtime telemetry and token statistics.

The table below summarizes Eve's core components and responsibilities:

Component	Typical Path / Service	Primary Responsibility	Design Implication
Agent Config	`agent/agent.ts`	Model selection, limits, experimental workflow config	Runtime strategy centralized, but identity comes from file paths, not an explicit name/id field.
Instructions	`agent/instructions.md` / `instructions.ts`	Persistent system prompt	Decouples "role/rules" from tool code.
Tools	`agent/tools/*.ts`	Executable actions with schemas	Filename is the tool name; tool results participate in checkpoint/replay.
Skills	`agent/skills/*`	On-demand procedural knowledge and workflows	Avoids stuffing long instructions into every prompt.
Channels	`agent/channels/*`	HTTP / Slack / Discord / custom webhook/WS entry	Same agent reusable across multiple channels.
Connections	`agent/connections/*`	MCP / OpenAPI / OAuth / API key integration	Moves URL, auth, and provider details out of prompts.
Subagents	`agent/subagents/*`	Focused delegation, concurrent fan-out, specialized identities	Supports fresh state and isolated tool surfaces.
Schedules	`agent/schedules/*`	Cron-driven tasks	Compiled into Cron Jobs on Vercel.
Sandbox	`agent/sandbox/*` / Vercel Sandbox	Filesystem, bash, isolated execution environment	Allows agent code execution/commands in an independent security context.
Runtime	Workflows + Functions	Session/turn orchestration, streaming, resume	Eve's production value concentrates here.

The following architecture diagram places these layers together for easier understanding of Eve's system boundaries. The connections are synthesized from official "runtime shape," "channels," "durability," and "Agent Runs" descriptions:

flowchart LR
    U[User or External System] --> C[Channel<br/>HTTP / Slack / Discord / GitHub / Custom]
    C --> S[Session / Turn Runtime]
    S --> W[Vercel Workflows<br/>checkpoint & replay]
    S --> F[Vercel Functions<br/>route + tool execution]
    S --> M[AI Gateway<br/>model routing]
    S --> T[Tools]
    S --> K[Skills]
    S --> G[Subagents]
    T --> X[Vercel Connect<br/>OAuth / API keys / MCP / OpenAPI]
    T --> B[Vercel Sandbox<br/>files / shell / code]
    S --> O[Agent Runs / OTel]
    C --> R[NDJSON Stream / continuationToken / sessionId]
    R --> U

Overall, Eve is an Agent framework that leans toward a backend systems framework rather than a simple prompt/tool SDK. It treats Agents as long-running, recoverable, auditable, and authorized service objects. This is the fundamental distinction from many "simplified agent loop" libraries.

Programming Model and API

Eve's public TypeScript API is unified: you primarily define agent capabilities through a set of define* helper functions, while identity comes from file paths, not a name field written in code. The official TypeScript API documentation explicitly states that tool agent/tools/get_weather.ts resolves to get_weather, and connection agent/connections/linear.ts resolves to linear. This means the file path itself is part of the API.

The most commonly used API surfaces include: defineAgent, defineTool, defineSkill, defineInstructions, defineChannel, eveChannel, defineMcpClientConnection, defineOpenAPIConnection, defineSchedule, defineHook, defineSandbox, defineEval, and the frontend binding useEveAgent. Official documentation also lists several non-define* helpers, such as always/once/never approval strategies, localDev/vercelOidc/placeholderAuth channel auth strategies, and default tool wrapping and disabling capabilities.

One frequently overlooked but architecturally critical point: Eve ships with a "default harness / default tool surface." From the official API listing, the framework's built-in wrappable or disableable default tools include bash, readFile, writeFile, glob, grep, webFetch, webSearch, todo, and loadSkill. This means Eve is not an "empty orchestrator" entering the runtime; it brings a strong default capability surface. Production setups must actively audit whether all these capabilities should be exposed to the model.

Here is a minimal, runnable Eve Agent. It corresponds to the official quickstart minimum structure: instructions.md + agent.ts + one typed tool.

// agent/agent.ts
import { defineAgent } from 'eve';

export default defineAgent({
	model: 'anthropic/claude-sonnet-5'
});

<!-- agent/instructions.md -->

You are a concise weather demo assistant.
If the user asks about weather, clearly state that the data here is mock data.

// agent/tools/get_weather.ts
import { defineTool } from 'eve/tools';
import { z } from 'zod';

export default defineTool({
	description: 'Returns mock weather data for a city.',
	inputSchema: z.object({
		city: z.string().min(1)
	}),
	async execute({ city }) {
		return {
			city,
			condition: 'Sunny',
			temperatureC: 24
		};
	}
});

These tools share key characteristics: inputs have schemas, return values must be JSON-serializable, and execution happens in the app runtime. The official tool documentation states that tools can import shared code from lib/, read process.env, and participate in Eve's durable pause/resume model; however, return values cannot be Dates, Maps, Sets, circular objects, or other unhandled structures.

Eve also provides a highly practical toModelOutput hook for trimming a tool's full output into a summary more suitable for the model, while keeping the full structure available for channels/UI/hooks. This design suits scenarios where "the UI needs rich data but the model only needs a summary," such as Slack card rendering, chart data, or large database result sets.

A simplified version looks like this:

// agent/tools/report_score.ts
import { defineTool } from 'eve/tools';
import { z } from 'zod';

export default defineTool({
	description: 'Generate a site score report.',
	inputSchema: z.object({
		domain: z.string()
	}),
	async execute({ domain }) {
		return {
			domain,
			score: 87,
			rawChecks: [
				{ name: 'seo', score: 90 },
				{ name: 'perf', score: 81 }
			]
		};
	},
	toModelOutput(output) {
		return {
			type: 'text',
			value: `Site ${output.domain} has a composite score of ${output.score}.`
		};
	}
});

Approvals are also a first-class concept in Eve's programming model. The official tool documentation provides approval strategies like always(), and release articles demonstrate conditional approval based on input criteria. The semantics are: the tool call point durably pauses the session, waits for human approval, then resumes from exactly where it left off. This is significantly stronger than "hanging a pending state in your own application code."

// agent/tools/refund_charge.ts
import { defineTool } from 'eve/tools';
import { always } from 'eve/tools/approval';
import { z } from 'zod';

export default defineTool({
	description: 'Process a refund.',
	inputSchema: z.object({
		chargeId: z.string(),
		amount: z.number().positive()
	}),
	approval: always(),
	async execute({ chargeId, amount }) {
		return { chargeId, refunded: amount };
	}
});

For frontend integration, Eve doesn't require you to separately maintain an agent service and a web service. The official frontend guides and templates show you can mount agents directly into web projects via eve/next, eve/sveltekit plugins, letting useEveAgent initiate sessions and streaming interactions under the same-origin /eve/v1/*; the Browser Agent Template also demonstrates withEve() mounting the agent within the same Next.js service.

Lifecycle, Event Streams, Concurrency, and State

Eve's runtime core concepts are sessions and turns. The official Concepts documentation defines these clearly: a session is a persistent conversation/task launched by a channel or HTTP request; each user message or external event creates a turn; within a turn, the agent can call tools, load skills, read/write sandbox files, delegate to sub-agents, and stream lifecycle events back to the client.

This is Eve's fundamental difference from traditional "request/response" chat interfaces. Both the official introduction article and documentation emphasize that an Eve session can stream-work, maintain durable state across multiple turns, pause on approvals/human replies, and resume in the future. The underlying mechanism relies on Vercel Workflows' event log plus deterministic replay: progress is persisted as an event log, and upon recovery, previously recorded steps are replayed to rebuild state. This means it can continue across cold starts, deployment switches, long waits for messages, or tool results.

Eve exposes two important identifiers externally: sessionId and continuationToken. The official README explains that sessionId is used for stream subscription and run inspection, while continuationToken is the recovery handle for sending the next message in the same conversation on that surface. This separation design helps distinguish "user-facing continuous sessions" from "platform-internal observable/recoverable objects."

The following event stream diagram synthesizes official session/streaming, subagent, approval, and channel documentation to show Eve's request path from entry to completion, or mid-way pause and recovery:

sequenceDiagram
    participant User as User/Platform Event
    participant Ch as Channel
    participant Rt as Eve Runtime
    participant Wf as Workflows
    participant Model as Model
    participant Tool as Tool/Sandbox/Connection
    participant Human as Approver

    User->>Ch: Message / webhook / slash command
    Ch->>Rt: Normalize input, start or resume session
    Rt->>Wf: Record turn checkpoint
    Rt->>Model: Inject instructions + skills + tools + history
    Model-->>Rt: Output text / tool calls / subagent calls
    Rt->>Tool: Execute tool or subagent
    Tool-->>Rt: Result
    Rt->>Wf: Record result, continue turn
    alt Approval Needed
        Rt->>Wf: Park session
        Rt-->>Human: Wait for approval
        Human-->>Rt: Approve/Reject
        Rt->>Wf: Resume session
    end
    Rt-->>Ch: NDJSON lifecycle event stream
    Ch-->>User: Text/card/message response

For concurrency, Eve's key capability comes from subagents. The official Subagents documentation states: the built-in agent tool delegates tasks to "a copy of the current agent"; if the model issues multiple agent calls in one response, Eve executes that batch concurrently and lets the parent agent continue after all return. This makes Eve ideal for "fixed small-batch task fan-out," such as querying multiple data sources in parallel, organizing multiple documents in parallel, or validating multiple hypotheses in parallel.

However, Eve's concurrency is not an "unlimited DAG scheduler." The official documentation explicitly states a default subagent depth limit of 3 child sessions, adjustable via limits.maxSubagentDepth; beyond this limit, the framework stops exposing further subagent tools to the model, and even if a call forcibly reaches the execution layer, it will be rejected. This indicates Eve's concurrency/recursion model is a controlled tree-like session expansion, not an arbitrary graph runtime.

For state management, Eve requires understanding in three layers. The first is durable session state: guaranteed by Workflows, persisting across turns and pause/resume. The second is sandbox filesystem state: built-in agent subagents share the sandbox with the parent agent, and file writes become immediately visible to the parent; declared subagents default to their own sandbox boundaries. The third is authored state / defineState context: the official documentation explicitly states that defineState is not shared between parent and child agents, and subagents always start from fresh durable state.

This means production setups must pay special attention to two boundary types:

First, shared sandboxes introduce concurrent write collision risk. The official documentation explicitly recommends providing non-overlapping write scopes for parallel child tasks. If you let multiple built-in agent subagents write to the same file or directory simultaneously, non-deterministic collisions are easy to encounter.

Second, declared subagents don't inherit root authored slots. Except for the built-in copy mode, a declared subagent won't automatically inherit the parent's instructions, tools, connections, skills, or sandbox; missing items fall back to framework defaults, not the root version. In other words, Eve's declared subagents are more like "local independent agent packages." This brings stronger isolation but requires you to explicitly replicate or extract shared capabilities.

Regarding replay and side effects, the official tool documentation emphasizes: completed steps do not re-run; Eve replays recorded results; but steps interrupted during execution will re-execute. Therefore any non-idempotent side effects — sending emails, charging fees, processing refunds, writing to external systems — must be designed idempotently or placed under approval/transactional protection. This is one of the most important traps in Eve production design.

Deployment, Scaling, Security, Observability, and Cost on Vercel

On the Vercel platform, Eve's deployment model is straightforward: the framework compiles agents into applications running on Vercel Functions, with Functions handling session requests, stream attachment, channel webhooks, and tool execution; while session/turn/subagent progress is persisted by Vercel Workflows. Official templates and frontend integration documentation further indicate that Eve can be combined with Next.js/SvelteKit web apps into a single project and single deployment, avoiding separate maintenance of agent services.

From a scaling perspective, Eve borrows Vercel platform capabilities rather than implementing its own cluster scheduler. The Concepts document states that agents run on Functions on Vercel, and because turns may run long and require incremental streaming output, Eve benefits from Fluid Compute enabled by default; meanwhile, what truly determines "sustained long execution" is Workflows' checkpoint/replay, not a long-lived process.

This also means Eve's scaling characteristics resemble a "serverless + durable orchestration" combination rather than traditional resident workers: short workloads handled by Functions throughput, long processes persistently saved and restored by Workflows. For production teams, this model typically brings stronger recovery and lower operational burden, but it also means you need to put performance analysis on dimensions like turn count, tool count, stream chunk count, and sandbox usage, rather than just looking at single-machine QPS.

Security and Permissions

Eve has several default security premises. First, model access goes through AI Gateway, which on Vercel can use OIDC instead of directly managing provider API keys; the AI Gateway documentation also states that every request requires authentication, supporting API keys or OIDC, and Bring Your Own Key.

Second, the sandbox as an isolation boundary is crucial. Vercel Sandbox documentation defines it as a compute primitive for safely running untrusted or user-generated code; Eve's release article further explains that agent-generated code doesn't enter the application runtime but executes in a sandbox within an independent security context; on Vercel, it uses ephemeral microVMs.

Third, the default HTTP channel is fail-closed. The official frontend guide writes clearly: if you don't customize agent/channels/eve.ts, the default registers eveChannel({ auth: [vercelOidc(), localDev()] }) — meaning it prioritizes accepting Vercel invocations and local development traffic, returning 401 for everything else. Only public demos and similar scenarios should explicitly use none() to relax authentication.

Additionally, Vercel Connect provides a solid credential governance foundation for Eve's external integrations. The official Connect documentation states that Connect supports Custom OAuth/OIDC + PKCE and API key storage; its "Concepts" documentation also describes triggers: third-party inbound webhooks are verified by Vercel Connect and forwarded to your project. This is particularly valuable for Slack, GitHub, SaaS webhooks, and multi-tenant integrations.

But Eve's official documentation also gives one very important warning: without additional stricter configuration, Eve may run under quite permissive default conditions, including tools executing directly without explicit approval requirements and sandbox network egress not being deny-all. Eve explicitly requires deployers to configure their own guardrails for approval strategies, tool limits, connection scopes, route/session authorization, sandbox controls, and telemetry exports.

Observability

Observability is where Eve looks most like a "productized backend framework." The official Observability documentation states that every Eve project gets Agent Runs by default: without a separate instrumentation.ts, you can see in the Vercel Dashboard runs broken down by trigger type, token input/output/cache volumes, turn counts, durations, and per-turn details including tool calls, reasoning, and timing.

If you need integration with your company's existing tracing体系, Eve also supports registering OpenTelemetry exporters in agent/instrumentation.ts. The official example demonstrates a Braintrust exporter and states that any OTel-compatible backend can be connected, such as Honeycomb, Datadog, Jaeger, etc. Since the instrumentation file runs automatically before agent startup and any agent code, it's also suitable for unified telemetry initialization.

Note that the official documentation simultaneously reminds: since Agent Runs captures session, reasoning, tool input/output, and other runtime information, if processing personal, sensitive, or regulated data, deployers may need to explain these collection behaviors in privacy materials and legal disclosures. In other words, the stronger the observability, the greater the compliance responsibility.

Cost Considerations

Eve is not a single billing item but a composite cost surface. The official Pricing document lists billable resources including: Vercel Functions, Vercel Workflows, Vercel Sandbox, AI Gateway, and underlying model providers. Common cost drivers include session/turn counts, prompt and output tokens, tool calls, stream writes to persistence, and sandbox execution/snapshot/network traffic.

The table below compresses Eve's main cost surfaces into a practical perspective:

Cost Surface	When It Increases	Typical Signal	Control Measures
Model tokens	Long history, large tool returns, long reasoning	Rising input/output/cached tokens in Agent Runs	Shorten history, trim tool output, load long flows into skills on-demand, use smaller models for low-risk tasks.
Workflow events/storage	Multi-turn, multi-stream chunks, multi-tool calls	High turn count, many stream chunks	Merge meaningless intermediate output, split long tasks, reduce meaningless streaming.
Functions duration	Slow external APIs, heavy tool logic	Elevated single-turn duration	Offload heavy CPU/untrusted tasks to sandbox; optimize external API calls.
Sandbox cost	Code writing/command running/snapshots/network access	Frequent sandbox usage	Enable sandbox only for necessary tasks, limit network policies and lifecycles.
Third-party service cost	SaaS APIs, MCP, databases	Rising external platform bills	Tool-level caching, idempotent retries, connection scope minimization.

A very practical suggestion: don't first calculate "roughly how much per month," but instead run a representative task profile. The official Pricing document recommends recording: turn count per task, model/tool call count per turn, token input/output/cache volume, stream data volume, and whether sandbox, snapshots, networking, and long-term persistence were used, then mapping to platform product billing items. This is a more reliable approach than "guessing by feel."

Integration Methods, Patterns, and Minimal Runnable Examples

Eve's integration methods can be summarized in three threads: channel access, external capability access, and application-layer persistence access. The official documentation shows that channels are entry adaptation layers, connections are external capability directories, and tools are arbitrary business logic executed in the app runtime; in other words, Eve's attitude toward "integration" is not giving you one super unified bus, but distributing integration to the appropriate layers.

Webhooks and Multi-Channel

The official Channels document defines a channel as a platform-to-agent edge adapter: it's responsible for normalizing platform input into user messages, holding the continuationToken, and deciding how responses are sent back. Besides the default HTTP channel, Eve has built-in platform channels for Slack, Discord, Teams, Telegram, Twilio, GitHub, and Linear, and allows custom webhook/WebSocket adaptation layers via defineChannel.

If you just want to expose the Agent as an HTTP API, you can use the officially provided /eve/v1/session and /eve/v1/session/:id/stream routes locally. The official documentation gives the simplest interaction method as follows:

curl -X POST http://127.0.0.1:3000/eve/v1/session \
  -H 'content-type: application/json' \
  -d '{"message":"Please summarize this repository"}'

# After getting x-eve-session-id from the response header:
curl http://127.0.0.1:3000/eve/v1/session/<sessionId>/stream

If you want to deploy the Web frontend and Agent as one project, the official frontend guides and templates demonstrate the simplest usage of useEveAgent. The following approach comes from the official SvelteKit guide; the React/Vue approach is the same: components only need to call useEveAgent(), without maintaining an independent Agent Host.

// Pseudocode, concept consistent with official Svelte example
import { useEveAgent } from 'eve/svelte';

const agent = useEveAgent();

await agent.send({
	message: 'Help me summarize the alert trends for the last 7 days'
});

Database and Application-Layer Persistence

Eve is not a database framework, but it integrates with databases easily. The official tool documentation explicitly states that tool execution happens in the app runtime, so it can import from lib/, read process.env, and connect to any database or backend. The official chat template documentation further demonstrates common practice: the browser interacts with the agent via same-origin /eve/v1/*, and streaming events are persisted to Postgres upon arrival, maintaining a session cursor for interruption recovery; the template stack explicitly uses Neon, Drizzle, Better Auth, and Upstash Redis.

This also explains Eve's best practice on databases: put strong transactions, record persistence, and session metadata storage at the application layer or tool layer, rather than expecting Eve's core to provide a universal ORM/memory store abstraction. This conclusion is a reasonable inference based on the design orientation of official templates and core documentation.

Also note a Vercel platform-level change: the official Postgres documentation states that Vercel Postgres is no longer offered, existing instances have been migrated to Neon, and new projects are advised to connect external Postgres through the Marketplace. This means when doing production databases with Eve on Vercel, you'll likely use Neon or external PG rather than a "platform-built-in database."

Third-Party APIs, MCP, and OpenAPI

Eve's preferred integration point for third-party capabilities is connections/. The release article states that a connection file can point to an MCP server or any OpenAPI-compatible API; Eve discovers remote tools, proxies authentication, and never exposes connection URLs and credentials to the model. The official TypeScript API also explicitly includes defineMcpClientConnection and defineOpenAPIConnection as two definition functions.

A minimal MCP connection example is almost a linear version of the official blog post:

// agent/connections/linear.ts
import { defineMcpClientConnection } from 'eve/connections';

export default defineMcpClientConnection({
	url: 'https://mcp.linear.app/sse',
	description: 'Linear workspace: issues, projects, cycles, comments.',
	auth: {
		getToken: async () => ({ token: process.env.LINEAR_API_TOKEN! })
	}
});

If you need user-level authorization without handling OAuth details yourself, leveraging Vercel Connect is more recommended. The official Connect documentation states it supports OAuth/OIDC with PKCE, API key storage, project links, and verified webhook triggers; while Eve documentation suggests pairing tools with Connect in delegated user/team credential scenarios.

Scheduled Tasks and Background Mode

Eve's scheduled task capability comes from agent/schedules/. Each schedule is a single file with a cron, which can be pure markdown task mode or a run({ receive, waitUntil, appAuth }) handler mode. The official documentation explicitly states: on Vercel, each schedule becomes a Vercel Cron Job, and cron expressions are interpreted in UTC.

Here is a minimal runnable handler schedule: it delivers a message to a Slack channel every minute.

// agent/schedules/heartbeat.ts
import { defineSchedule } from 'eve/schedules';
import slack from '../channels/slack.js';

export default defineSchedule({
	cron: '* * * * *',
	async run({ receive, waitUntil, appAuth }) {
		waitUntil(
			receive(slack, {
				message: 'Check if there are new critical alerts; if not, send nothing.',
				target: { channelId: 'C0123ABC' },
				auth: appAuth
			})
		);
	}
});

A very key limitation: markdown task-mode schedules cannot park to wait for approval or OAuth login; only the handler form can continue with durable runtime semantics after handoff. If planned tasks might encounter human approval or user authorization in production, this must be designed upfront.

Common Usage Patterns

Combining official templates and documentation, I believe Eve is best suited for the following patterns:

Pattern	Why It Fits Eve	Representative Resources
Web Chat + Durable Backend	Same-origin `/eve/v1/*`, co-deployed frontend/backend, recoverable sessions	Eve Chat Template, Browser Agent Template
Slack/Team Assistant	Channels are first-class concepts, support cards and approvals	Slack starter, content assistant
Approval-Driven Operations/Internal Tools	Tool approvals durably pause/resume	Tools approval docs, release article
Browser/Code/File Workflows	Each agent ships with a sandbox, suitable for ad-hoc scripts and file artifacts	Official release article, Browser Agent Template
Scheduled Inspection/Daily Reports/Auto Triage	Schedules compile directly into Vercel Cron Jobs	Schedules documentation

Comparison with Alternative Frameworks

As of this writing, there is no official standardized benchmark comparing Eve with other mainstream Agent frameworks. Therefore the following "performance/throughput" assessments are not lab benchmarks but qualitative analyses based on each framework's official documentation describing execution models, persistence semantics, deployment methods, and built-in capabilities. I will explicitly mark these as architectural inferences rather than official benchmarks.

Overall Comparison

Dimension	Eve	LangGraph	OpenAI Agents SDK	Mastra
Design Center	Filesystem-first, agent is a directory; production capabilities built-in	Graph/state-machine-first, low-level orchestration runtime	SDK-first, agents/runs/handoffs/sessions defined in code	TypeScript AI application framework, agents/workflows/memory equally emphasized
Durable Execution	Built-in, based on Vercel Workflows	Strong suit,主打 durable execution/persistence	SDK manages turns/tools/sessions; also supports sandbox agents	Has durable agents, but docs still label Beta
Concurrency & Multi-Agent	Built-in subagents, folder-style specialists, default depth limits	Most flexible for complex orchestration with graph nodes and state transitions	Strong handoffs/specialists, ideal for OpenAI-native runtimes	Composable workflows, memory, multi-agent patterns
Channels/Multi-Surface	Built-in HTTP + multi-platform channels + custom channels	Typically requires self-integration of Web/Slack surfaces	Primarily SDK/runtime semantics, channel layer needs app-level integration	More app-framework oriented, deploys anywhere, but channels aren't a first-class abstraction like Eve
Platform Binding	Strong, Vercel-native best	Relatively neutral	Most friendly to OpenAI ecosystem, but can pair with other providers	Deploys anywhere, stronger framework neutrality
Developer Experience	Very efficient for Vercel users; clear directory structure; many conventions	Flexible but lower-level, higher learning curve	Direct and natural for SDK users; more code-centric	Strong TypeScript full-stack feel; broad feature surface, but system boundaries more dispersed

Design Philosophy and Developer Experience

Eve vs LangGraph: LangGraph officially positions itself as a low-level orchestration framework and runtime, focusing on durable execution, streaming, HITL, and persistence; while Eve is more of a "framework-definition product" that brings conventions, file layout, channels, and platform integrations together. If you want precise control over state graphs, nodes, back-edges, and custom scheduling, LangGraph is freer; if you want to quickly deploy a multi-channel, recoverable, sandbox-equipped agent, Eve's path is shorter.

Eve vs OpenAI Agents SDK: The official OpenAI documentation clearly distinguishes the Responses API from the Agents SDK, the latter being suitable when applications need the SDK to handle orchestration, tool execution, approvals, state, and sessions. It's a very "SDK/code-first" design, while Eve is "directory/framework-first." If your capabilities heavily depend on OpenAI-native tools, container execution, realtime/voice agents, and the tracing ecosystem, OpenAI Agents SDK's integrated experience will be strong; but if you value Web/Slack/GitHub multi-entry unification, Vercel deployment, and platform governance more, Eve is more convenient.

Eve vs Mastra: Mastra officially emphasizes itself as an open-source TypeScript AI framework providing agents, workflows, memory, workspaces, and observability, deployable to various frameworks and standalone servers. In contrast, Eve has a narrower framework boundary and stronger platform binding, but does "condensing production Agent shapes into a directory" more thoroughly. Mastra is attractive for teams wanting to unify agents + workflows + memory + apps; Eve is better suited for treating "durable agent backends" as separate product objects to govern.

Performance and Runtime Trade-offs

If looking only at single-turn, short-path, light-tool scenarios, Eve's Workflow persistence, stream writes, and Agent Runs collection bring additional control surface costs; in such scenarios, using the Responses API directly, OpenAI Agents SDK, or lightweight LangGraph graphs could theoretically have lower runtime overhead. This is an architectural inference based on official descriptions of Eve's Workflow persistence, stream persistence, and multi-layer billing resources.

But if the task is long-running, interruptible, requiring approvals, needing multi-turn recovery, needing sandboxes or subagent fan-out, Eve's built-in durable model often offsets the complexity these additional overheads bring: you don't need to implement replay, pause/resume, stream reconnection, auditing, and channel glue code yourself. In other words, Eve's advantage isn't "fastest," but "making high-complexity production semantics the default."

When to Choose Which

Your Primary Need	Recommendation
I'm on Vercel, need a shippable durable agent backend with Web/Slack/approvals/observability	Eve
I need fine-grained control over state graphs, branching, and custom orchestration	LangGraph
I deeply use OpenAI platform-native tools/sandbox/realtime	OpenAI Agents SDK
I want a TypeScript full-stack AI framework with agents/workflows/memory unified	Mastra

Limitations, Pitfalls, Troubleshooting, Ecosystem, and Production Adoption Advice

Known Limitations and Common Pitfalls

Eve is still Beta/Preview. This means APIs, behaviors, and documentation may change before GA. If your organization has strict requirements for platform stability, long-cycle support, or compliance auditing, Eve should be treated as "production-capable pilot, not zero-constraint full rollout."
Strong platform binding. Although official documentation includes self-host / eve build && eve start instructions, Eve's smoothest capability composition clearly occurs on the Vercel platform: AI Gateway OIDC, Functions, Workflows, Sandbox, Connect, and Agent Runs are all key components. Running outside Vercel is possible but increases mental model and operational complexity. This judgment comes from official product combinations and Vercel-specific integration documentation.
Large default capability surface. The official documentation explicitly lists default tools including bash, file read/write, webFetch, webSearch, todo, loadSkill, and the warning that "without additional controls, tool execution may be permissive and sandbox networking may not be deny-all." Therefore Eve cannot rely solely on instructions hoping the model "won't misbehave"; you must close the aperture from four layers simultaneously: auth, approval, tools, connections, and sandbox policy.
Concurrent fan-out easily introduces shared write collisions. If you run multiple subtasks in parallel with the built-in agent tool, they share the parent agent's sandbox and tool surface, and writing the same path causes non-deterministic overwrites. The official documentation explicitly recommends providing non-overlapping write scopes.
Non-idempotent tools may re-run after interruption. The official tool documentation has explained this very clearly. In production, charging, notifications, writing to external systems must introduce idempotency keys, state locks, or approvals.
Two typical pitfalls with scheduled tasks: Vercel Cron interprets expressions in UTC; and markdown task-mode schedules cannot wait for approval/OAuth. Additionally, custom hosting without properly starting the Nitro schedule runner may result in "tasks compiled but not auto-triggered."
Subagent naming conflicts and depth limits. Subagent tool names share the same runtime tool namespace as regular tools; if subagents/researcher and tools/researcher.ts have the same name, the build will fail; the default depth limit is also 3 child sessions.
Limited official Chinese resources. As of this writing, Vercel's official Eve documentation and Knowledge Base are primarily in English; Chinese content mostly comes from community interpretations and reposts with varying quality. Chinese-language teams should treat official English docs + repository docs as the primary source, with Chinese content as supplementary material.

Troubleshooting Guide

Problem Symptom	Priority Check	Basis
New files not recognized by Eve	Run `eve info`, check discovery diagnostics and `.eve/` artifacts; confirm file path is in the correct slot	Official Project Layout recommends `eve info` for discovery failures
Browser frontend gets 401	Check `agent/channels/eve.ts` auth; default is `vercelOidc()` + `localDev()`, not public	Official frontend guide states default eve channel is fail-closed
Subagent can't access parent tools/skills	Expected behavior for declared subagents; they don't inherit root authored slots	Official Subagents documentation
Tool return value causes exceptions	Check if output is JSON-serializable; avoid Date/Map/Set/NaN/circular objects	Official Tools documentation
Scheduled tasks don't trigger in local dev	`eve dev` doesn't auto-trigger on cron; use dev dispatch route or built app	Official Schedules documentation
Schedule doesn't execute in self-hosted production	Confirm whether you actually ran `eve start`/Nitro scheduler, not just provided HTTP	Official Schedules documentation
Sensitive actions still potentially callable	Subagent isolation is not an approval boundary; use approval, connection approval, route/session auth	Official Subagents and safeguards documentation

Ecosystem, Maturity, and Recommended Resources

Eve's ecosystem is rapidly forming. Official and official-related resources currently include at minimum: core repository, Vercel Docs, Vercel Blog/Changelog, Knowledge Base, and multiple open-source templates. The GitHub repository has approximately 3.2k stars and 252 forks as of this writing, with active signals in Issues, PRs, and Discussions.

Recommended resources, read in "official first, then templates, then community" priority order:

Resource	Purpose	Priority
Eve Overview Docs	Framework overview, platform mapping, minimal example	Very High
Concepts	Core mental model for sessions/turns, durability, channels, sandbox, subagents	Very High
TypeScript API	Authoritative entry for define*, ctx, import paths	Very High
Project Layout	File slots, root/subagent boundaries, debugging entry	Very High
Tools / Subagents / Schedules Docs	Production-grade feature details and pitfalls	Very High
Observability / Pricing Docs	Core basis for monitoring and cost control post-launch	Very High
Official Release Article: Introducing Eve	Most comprehensive design philosophy and end-to-end examples	Very High
Eve Chat Template	Best practice for Web Chat + Postgres persistence	Very High
Eve Slack Agent / Content Agent Templates	Team assistants, content workflows, Connect/OAuth integration patterns	High
Browser Agent Template	Web Chat + browser agent + single-project deployment paradigm	High
Chinese Community Interpretations	Supplementary understanding, but confirm back to official docs	Medium

Production Adoption Recommendations

If you're asking "is Eve production-ready," my conclusion is: yes, but on the premise that you accept it as a Vercel-native, strongly opinionated, still-Beta production Agent framework. For most teams already deeply using Vercel, I recommend evaluating Eve as the first-priority pilot framework, because its integration of durable execution, sandbox, channels, approvals, observability, and deployment is genuinely high.

But "pilot" doesn't mean "directly replace everything." The safest path for production adoption:

Stage	Recommended Approach
Pilot	Start with a single channel and single business line, e.g., internal Slack assistant, Web Chat retrieval assistant, PR triage
Governance	Define auth, approval, sandbox egress, tool allowlist first; then write model instructions
Measurement	Use Agent Runs as the primary lens for tokens, turns, tools, duration; integrate OTel if needed
Stabilization	Add evaluations, CI, idempotency, and cost alerts; incorporate schedules and subagents deeply into operational boundaries

Migration Checklist

If your existing system comes from LangGraph, custom agent loops, OpenAI Agents SDK, or other frameworks, here is a pragmatic migration checklist:

Check Item	Migration Action
Agent identity and directory structure	Map "roles, tools, skills, channels, scheduled tasks, subagents" to file slots under `agent/`
Tool functions	Wrap existing tools with `defineTool`, add zod schemas, confirm output is JSON-serializable
Permissions and sensitive operations	Add approval or route/session auth for refunds, email sending, DB writes, production API calls
Prompts/knowledge	Split long instructions into `instructions.md` + `skills/`; avoid context bloat
External APIs	Migrate to `connections/` + Connect at priority; at minimum strip credentials from prompt/tool text
Long flows/multi-turn recovery	Migrate logic previously maintained by custom state machines to session/turn thinking; identify pause/resume boundaries
Web/Slack entry points	Prefer built-in channels; use `defineChannel` only for custom surfaces
Frontend deployment	If already using Next.js/Nuxt/SvelteKit, prioritize single-project mounting over dual services
Scheduled tasks	Clarify UTC cron, whether审批/OAuth may trigger, whether handler mode is needed
Idempotency and rollback	Design idempotency keys and compensation logic for all external side effects, since interrupted steps may re-run
Observability and privacy	Enable Agent Runs, integrate OTel if needed, and assess sensitive data tracking disclosure obligations

Final Recommendation

If your tech stack is centered around Vercel, my recommendation is clear: you can adopt Eve, but proceed as a "production framework pilot" rather than a "prototype tool." Prioritize landing in internal assistants, content workflows, retrieval/knowledge assistants, PR triage, and browser/file-type agents; for financial, legal, medical, and external write-operation scenarios, must-frontload approvals, idempotency, auditing, and permissions as design constraints rather than post-launch patches. Comprehensive official documentation, templates, and platform capabilities indicate that Eve's greatest value isn't making Agents "smarter," but making Agents more like genuinely operable backend systems.

Loop Engineering Harness：AI Agent 架构中的迭代引擎

Tue, 16 Jun 2026 00:00:00 GMT

一、概念定义：什么是 Loop Engineering Harness

Loop Engineering Harness（循环工程 harness）是 AI Agent 架构中的核心控制框架，它定义了 Agent 在执行复杂任务时的迭代循环机制——从感知环境、制定计划、执行动作到评估结果的完整闭环。

与传统的线性流水线不同，Harness 强调的是有状态的持续迭代：Agent 不是一次性输出结果，而是在多轮循环中不断修正、优化，直到满足终止条件。这种设计直接借鉴了软件工程中的"Harness"概念——为被测系统提供运行时环境、输入注入和结果捕获的基础设施。

在 AI Agent 语境下，Harness 承担三个关键职责：

循环控制：决定何时继续迭代、何时终止
状态管理：维护跨轮次的上下文和记忆
反馈整合：将执行结果转化为下一轮迭代的输入

二、设计原理：为什么需要循环工程

2.1 复杂任务的本质需求

现实世界的任务很少能在单次推理中完成。以代码生成为例：Agent 需要理解需求、编写代码、运行测试、修复错误、优化性能——每一步都可能产生新的信息，改变后续决策。

传统线性流程：输入 → 处理 → 输出（无反馈）
Harness 循环：输入 → 计划 → 执行 → 评估 → 反馈 → 再计划 → ...

2.2 核心设计原则

原则	说明
有界迭代	设置最大循环次数，防止无限循环
状态可追溯	每轮状态可序列化，支持断点恢复
反馈驱动	执行结果直接影响下一轮决策
可观测性	循环过程对开发者透明，便于调试

2.3 终止条件设计

Harness 必须明确定义循环退出的条件，常见策略包括：

目标达成：任务指标满足预设阈值
预算耗尽：达到最大迭代次数或 token 上限
收敛检测：连续多轮输出无显著变化
人工干预：外部信号强制终止

三、核心组件架构

3.1 组件总览

┌─────────────────────────────────────────────┐
│              Loop Engineering Harness         │
│                                               │
│  ┌──────────┐  ┌──────────┐  ┌────────────┐ │
│  │  Planner  │→ │ Executor │→ │ Evaluator  │ │
│  │  (规划器) │  │ (执行器) │  │ (评估器)   │ │
│  └──────────┘  └──────────┘  └────────────┘ │
│       ↑                            │          │
│       └──────── 反馈回路 ──────────┘          │
│                                               │
│  ┌──────────────────────────────────────────┐ │
│  │         State Manager (状态管理器)         │ │
│  └──────────────────────────────────────────┘ │
│  ┌──────────────────────────────────────────┐ │
│  │       Termination Judge (终止判断器)       │ │
│  └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────┘

3.2 各组件详解

Planner（规划器）：根据当前状态和目标，生成下一步行动计划。它接收来自 Evaluator 的反馈，动态调整策略。在实现中，Planner 通常是一个 LLM 调用，prompt 包含历史上下文和上一轮评估结果。

Executor（执行器）：将 Planner 产出的计划转化为具体动作——调用工具、执行代码、查询数据库或生成内容。Executor 是 Agent 与外部世界交互的桥梁。

Evaluator（评估器）：对执行结果进行质量评估，判断是否达成子目标，识别错误和偏差。评估可以是规则化的（如测试用例通过率）或模型驱动的（如 LLM-as-Judge）。

State Manager（状态管理器）：维护整个循环的状态——包括任务进度、历史动作、中间结果和上下文窗口。它是实现"有状态迭代"的关键。

Termination Judge（终止判断器）：基于预设规则和当前状态，判断是否应该退出循环。

四、在 AI Agent 架构中的关键作用

4.1 从"工具调用"到"自主迭代"

早期的 Function Calling 模式是单轮的：模型决定调用哪个工具，获取结果后输出。Harness 将其升级为多轮自主迭代，使 Agent 具备真正的"尝试-失败-重试"能力。

4.2 工程化价值

维度	无 Harness	有 Harness
可靠性	单次失败即终止	自动重试与修正
可调试性	黑盒输出	每轮状态可追溯
可控性	无法干预	支持人工介入点
可扩展性	硬编码流程	模块化组件替换

4.3 典型应用场景

自动化测试修复：Agent 编写测试 → 运行 → 分析失败 → 修复代码 → 重跑
数据分析管道：探索数据 → 发现异常 → 清洗 → 重新分析
文档生成：起草 → 审查 → 修订 → 定稿

五、最小化实现示例

以下是一个简化的 Loop Engineering Harness 实现，展示核心循环逻辑：

interface AgentState {
	goal: string;
	history: Array<{ action: string; result: string }>;
	iteration: number;
	maxIterations: number;
}

interface HarnessResult {
	success: boolean;
	finalState: AgentState;
	output: string;
}

class LoopEngineeringHarness {
	constructor(
		private planner: Planner,
		private executor: Executor,
		private evaluator: Evaluator,
		private stateManager: StateManager
	) {}

	async run(initialState: AgentState): Promise<HarnessResult> {
		let state = this.stateManager.initialize(initialState);

		while (state.iteration < state.maxIterations) {
			// 1. 规划下一步
			const plan = await this.planner.plan(state);

			// 2. 执行计划
			const result = await this.executor.execute(plan);

			// 3. 评估结果
			const evaluation = await this.evaluator.evaluate(result, state.goal);

			// 4. 更新状态
			state = this.stateManager.update(state, {
				action: plan.action,
				result: result.output,
				evaluation
			});

			// 5. 终止判断
			if (evaluation.goalAchieved || evaluation.shouldTerminate) {
				return {
					success: evaluation.goalAchieved,
					finalState: state,
					output: result.output
				};
			}

			state.iteration++;
		}

		return { success: false, finalState: state, output: '达到最大迭代次数' };
	}
}

六、总结

Loop Engineering Harness 是 AI Agent 从"单次响应"走向"持续自主执行"的架构基石。它通过规划-执行-评估的闭环设计，赋予 Agent 处理复杂、开放式任务的能力。在工程实践中，Harness 的质量直接决定了 Agent 的可靠性上限——好的 Harness 能让中等能力的模型完成出色的工作，而缺乏 Harness 约束的强模型也可能在复杂任务中迷失方向。

构建 Harness 的核心不在于追求更多的迭代轮次，而在于设计精准的反馈信号和合理的终止策略——让 Agent 知道何时该坚持、何时该放弃、何时该求助。

沙箱（Sandbox）技术完全指南

Tue, 16 Jun 2026 00:00:00 GMT

沙箱技术是现代网络安全和软件开发中的重要工具，用于在隔离环境中运行潜在危险的程序，以防止其对系统造成损害。

一、沙箱核心概念

**沙箱（Sandbox）**是一种安全隔离技术，通过创建一个受限制的执行环境，将潜在不安全的程序、代码或数据与系统核心部分隔离，防止其对系统或用户数据造成破坏。

1.1 什么是沙箱

沙箱就像一个透明的、坚不可摧的玻璃箱。在这个玻璃箱里运行可疑程序（如来路不明的软件、可疑邮件附件、网页脚本），无论程序在里面做什么（修改文件、删除数据、连接网络），都无法接触到真实的宿主系统。

沙箱的核心价值在于：

安全隔离：不可信代码无法影响宿主系统
行为监控：完整记录程序的所有操作行为
资源限制：防止恶意程序耗尽系统资源
环境还原：执行结束后可快速恢复干净状态

1.2 沙箱整体架构

沙箱系统采用三层架构设计：

宿主系统层：提供底层硬件和操作系统支持
沙箱隔离层：实现资源隔离、权限控制、权限检查、行为监控
沙箱环境层：提供隔离的虚拟文件系统、虚拟网络、虚拟注册表

二、沙箱核心原理

2.1 工作原理详解

沙箱的完整工作流程：

程序输入：待分析/执行的程序进入沙箱系统
隔离环境：程序被放置在隔离的执行环境中
API拦截：所有系统调用被拦截和检查
行为监控：记录程序的所有API调用和行为
日志记录：生成完整的行为日志
安全分析：基于行为数据进行安全评估

2.2 核心技术原理

资源隔离

沙箱为程序分配独立的虚拟资源：

内存空间：独立的地址空间，防止越界访问
文件系统：虚拟文件系统，所有修改仅在沙箱内可见
网络接口：虚拟网络，可限制或监控网络访问
注册表：虚拟注册表，Windows系统专用

权限控制

通过严格的权限策略限制程序行为：

禁止访问敏感系统文件
限制进程创建和注入
控制网络访问范围
禁止修改系统配置

行为监控

通过Hook技术拦截并记录所有关键操作：

文件读写操作
注册表修改
网络连接建立
进程/线程创建
系统API调用

三、沙箱技术实现方式

3.1 主流实现方案对比

技术方案	隔离强度	性能损耗	启动速度	内存开销	代表产品
虚拟机 (VM)	⭐⭐⭐⭐⭐ 最强	高 (~30%)	慢（分钟级）	大（GB级）	VMware, VirtualBox
容器 (Docker)	⭐⭐⭐ 中等	低 (~5%)	快（秒级）	小（MB级）	Docker, LXC
用户态内核 (gVisor)	⭐⭐⭐⭐ 强	中 (~15%)	较快	中等	gVisor, Kata
系统调用拦截	⭐⭐ 较弱	极低 (~2%)	极快	极小	Sandboxie, seccomp

3.2 各方案技术细节

1. 虚拟机方案

原理：通过Hypervisor模拟完整的硬件环境，运行独立的Guest OS

核心技术：硬件辅助虚拟化（Intel VT-x, AMD-V）

优点：隔离最彻底，几乎无法逃逸

缺点：资源开销大，启动慢

2. 容器方案

原理：基于Linux Namespace和Cgroups实现进程级隔离

核心技术：

PID Namespace - 进程隔离
NET Namespace - 网络隔离
MOUNT Namespace - 文件系统隔离
Cgroups - 资源限制

3. 用户态内核方案

原理：在用户空间实现一个迷你内核（Sentry），拦截所有系统调用

代表：Google gVisor

优势：在安全性和性能间取得最佳平衡

4. 系统调用拦截方案

原理：通过API Hook或seccomp过滤危险系统调用

代表：Sandboxie, Windows Sandbox

优势：轻量快速，用户体验好

四、沙箱制作方法

4.1 沙箱搭建流程

4.2 基于Docker的简易沙箱制作

步骤1：创建Dockerfile

FROM python:3.11-slim

# 创建非特权用户
RUN groupadd -r sandbox && useradd -r -g sandbox sandboxuser

# 设置工作目录
WORKDIR /sandbox

# 复制执行脚本
COPY run.sh .
RUN chmod +x run.sh

# 切换到非特权用户
USER sandboxuser

# 入口命令
CMD ["./run.sh"]

步骤2：编写执行脚本run.sh

#!/bin/bash
set -e

# 资源限制
ulimit -n 1024      # 文件描述符限制
ulimit -u 64        # 进程数限制
ulimit -f 10240     # 文件大小限制

# 执行用户代码
python3 user_code.py

步骤3：构建并运行

# 构建镜像
docker build -t python-sandbox .

# 运行沙箱（自动清理）
docker run --rm \
  --memory=512m \
  --cpus=0.5 \
  --pids-limit=64 \
  --network=none \
  -v $(pwd)/user_code.py:/sandbox/user_code.py:ro \
  python-sandbox

4.3 基于Linux Namespace的原生沙箱

#define _GNU_SOURCE
#include &lt;sched.h&gt;
#include &lt;stdio.h&gt;
#include &lt;stdlib.h&gt;
#include &lt;sys/wait.h&gt;
#include &lt;unistd.h&gt;

#define STACK_SIZE (1024 * 1024)
static char child_stack[STACK_SIZE];

static int child_func(void *arg) {
    printf("Sandbox: PID = %d\n", getpid());
    // 在隔离环境中执行程序
    execl("/bin/bash", "/bin/bash", NULL);
    return 0;
}

int main(int argc, char *argv[]) {
    // 创建隔离命名空间
    int flags = CLONE_NEWUTS | CLONE_NEWIPC | CLONE_NEWPID |
                CLONE_NEWNS | CLONE_NEWNET | CLONE_NEWUSER;

    pid_t child_pid = clone(child_func, child_stack + STACK_SIZE,
                            flags | SIGCHLD, NULL);

    waitpid(child_pid, NULL, 0);
    return 0;
}

五、沙箱使用场景

5.1 网络安全领域

恶意代码分析是沙箱最经典的应用场景

病毒/木马分析：在隔离环境中运行恶意软件，观察其行为
勒索软件研究：分析加密算法和传播机制
APT攻击溯源：还原完整攻击链
杀毒软件动态检测：360、卡巴斯基等均内置沙箱

5.2 软件开发与测试

代码安全执行：AI Agent生成的代码在沙箱中运行验证
兼容性测试：模拟不同环境测试软件
CI/CD流水线：隔离的构建和测试环境
漏洞复现：安全研究人员验证PoC

5.3 AI Agent安全执行

AI Agent沙箱是当前最热门的应用方向

代码解释器：ChatGPT、Claude的代码执行环境
工具调用安全：防止Agent执行危险操作
多Agent协作：每个Agent独立沙箱隔离
数据处理：用户上传文件的安全处理

5.4 浏览器安全

Chrome Site Isolation：每个站点独立进程
网页沙箱：JavaScript执行环境隔离
下载文件扫描：自动在沙箱中打开可疑文件

5.5 其他场景

教育实训：学生实验环境，防止误操作
隐私计算：数据可用不可见
游戏反作弊：检测游戏内存修改
沙箱浏览器：隔离访问可疑网站

六、主流沙箱工具介绍

6.1 恶意代码分析沙箱

Cuckoo Sandbox（开源）

定位：工业级自动化恶意软件分析系统

特点：

支持Windows、Linux、macOS、Android
完整记录API调用、文件、网络、注册表
生成详细分析报告和IOC
开源免费，可自定义扩展

Any.Run（在线）

定位：交互式在线沙箱

特点：

实时交互，像操作真实电脑一样
支持视频录制和行为回放
社区分享分析结果
免费版有功能限制

Hybrid Analysis

定位：免费在线恶意软件分析平台

特点：

集成CrowdStrike威胁情报
支持VxStream沙箱引擎
详细的行为报告和IOC提取

6.2 桌面端沙箱工具

Sandboxie-Plus（开源）

定位：Windows经典沙箱软件

特点：

轻量级，几乎不影响性能
右键菜单"在沙箱中运行"
支持多沙箱独立配置
开源免费，社区活跃

Windows Sandbox（系统内置）

定位：Windows 10/11 Pro内置轻量沙箱

特点：

系统原生，无需额外安装
每次启动都是干净环境
关闭后自动销毁所有数据
基于Hyper-V技术

Shadow Defender

定位：影子系统软件

特点：

整个磁盘进入影子模式
所有修改重启后还原
适合测试软件和系统防护

6.3 容器化沙箱方案

Docker

定位：最流行的容器化平台

沙箱应用：快速创建隔离的执行环境

gVisor

定位：Google开源的容器运行时

特点：用户态内核，比Docker更安全

Kata Containers

定位：轻量级虚拟机容器

特点：VM级安全，容器级性能

6.4 AI Agent专用沙箱

e2b

定位：AI Agent云沙箱服务

特点：专为代码解释器设计，支持长会话

Cube Sandbox

定位：高并发AI Agent执行沙箱

特点：支持千级Agent并行执行

七、沙箱安全与逃逸

注意：没有100%安全的沙箱，理解其局限性很重要

7.1 常见沙箱逃逸技术

内核漏洞利用：通过内核漏洞突破隔离
容器逃逸：CVE-2019-5736等经典漏洞
侧信道攻击：通过时序等侧信道泄露信息
硬件漏洞：Spectre、Meltdown等CPU漏洞

7.2 沙箱加固建议

多层隔离：容器+虚拟机双重防护
最小权限：遵循最小权限原则
及时更新：修复已知安全漏洞
行为审计：完整记录所有操作
网络隔离：默认禁止网络访问

八、总结与展望

沙箱技术作为数字世界的"安全试炼场"，在网络安全、软件开发、AI安全等领域发挥着不可替代的作用。随着AI Agent的爆发式增长，沙箱技术正在迎来新的发展机遇：

高性能：在保证安全的前提下进一步降低性能损耗
高并发：支持大规模AI Agent并行执行
智能化：结合AI自动分析和响应威胁
标准化：形成统一的沙箱接口标准

参考资料：

Linux Namespace和Cgroups官方文档
gVisor官方技术白皮书
Cuckoo Sandbox开源项目文档
Docker容器安全最佳实践
Windows Sandbox技术原理说明

Building WishLive: A Multi-Agent Runtime

Sun, 14 Jun 2026 00:00:00 GMT

The Idea

Most agent frameworks treat agents like stateless functions. Call in, response out. No memory, no runtime, no economics.

I wanted something different: a runtime where agents are first-class actors. They register, discover each other, negotiate terms, execute work, and settle payments — all inside an event-driven system with observable state.

This is the story of building that system in 48 hours for a hackathon.

Architecture: Six Layers, One Event Bus

The system is structured as six layers, each independently designed but communication-coupled through an event bus:

Every layer publishes to typed event streams. The event log is the source of truth — no hidden state.

Layer 1: Registry — The Agent Internet

The registry is the DNS + service discovery for agents. Each agent registers with a typed card:

interface AgentCard {
	agent_id: string;
	type: 'musician' | 'venue' | 'manager' | 'audience' | 'business';
	name: string;
	skills: string[];
	reputation: number;
	// ...
}

Agents send heartbeats every 60 seconds. If they miss, the registry marks them OFFLINE. The discover() method implements an A2A discovery protocol — one agent asks a manager agent to find matching candidates by skill, genre, city, or capacity.

The system seeds 57 agents across 6 types on startup. The onlineCount() method tracks availability by type in real time.

Key runtime behavior: ensureSeeded() auto-seeds on first access, so every service that depends on the registry gets a populated agent network without explicit initialization order.

Layer 2: Wish → Workflow Pipeline

Users submit wishes: "I want to see Artist X in City Y on Date Z." The WishWorkflowService:

Aggregates wishes by cohort key (genre + city)
Runs an agent to record the wish processing
When a cohort reaches threshold (10 wishes), triggers demand creation
The matching engine finds candidate musicians and venues from the registry
Publishes demand events that drive the rest of the system

The threshold is a deliberate design choice — it prevents premature negotiation on insufficient demand.

Layer 3: Runtime — The Execution Engine

This is the core. AgentRuntimeService.run() creates a session, then executes in a loop:

session started → agent thinks → tool execution → message → session completed

Every step publishes to agent.runtime event stream · Langfuse telemetry on all sessions

The runtime supports dual-mode execution:

Real mode: Uses the Vercel AI SDK with generateText(), passing runtime tools for discovery, pricing, proposals. The agent decides which tools to call based on the user message.
Simulated mode: Deterministic fallback when no AI provider is configured. Executes the same tool plan but returns structured mock responses.

The graceful degradation is intentional — simulated mode means I can demo the entire system without an API key. The switch happens transparently at the session level:

if (mode === 'real') {
	try {
		result = await generateWithAISDK(agent, input, session);
	} catch (error) {
		mode = 'simulated'; // graceful fallback
		result = await executePlannedTools(agent, input, session, mode);
	}
} else {
	result = await executePlannedTools(agent, input, session, mode);
}

Every step publishes to the agent.runtime event stream — thoughts, tool calls, messages, session lifecycle. Langfuse telemetry is wired in for production observability.

Layer 4: Negotiation — Stateful Agent Coordination

The negotiation service models multi-agent coordination as explicit, stateful protocols:

create → propose → counter → accept → deal
                    ↓
                 reject

Each negotiation has a workflowId, conversationId, and tracks all proposals. The runAutonomousNegotiation() method runs the full pipeline automatically:

AUTONOMOUS NEGOTIATION PROTOCOL

Musician Agent Registry Venue Agent

or → reject → negotiation ends

Matches top musician and venue candidates from demand
Musician sends initial proposal with terms (venue fee, split percentage)
Venue counters with adjusted terms
Musician accepts
A Deal is created, ready for settlement

The protocol is general enough for human-in-the-loop override — each step has API endpoints for manual accept/reject/counter.

Layer 5: Settlement — Economics On-Chain and Off

Settlement is where execution becomes real. The system supports dual settlement modes:

On-chain (production):

Solidity smart contracts: Escrow (lock/release funds), TicketNFT (mint event tickets), AgentProfile (agent identity)
Hardhat deployment to local or testnet
Funds held in escrow until deal confirmed

Local (development):

Deterministic hash-based transaction IDs
localTxHash() generates consistent 0x-prefixed hashes
Full simulation of the settlement flow without a blockchain

SETTLEMENT PIPELINE

All settlement actions require human confirmation before execution

The critical design choice: human confirmation gates. Every settlement action requires explicit confirmation:

async createEscrow(input, options: { confirmed: boolean }) {
  if (!options.confirmed) {
    throw new SettlementError(409, "Human confirmation required");
  }
  // ...
}

This prevents autonomous agents from moving real money without oversight.

Layer 6: Concierge — The Observability Layer

The concierge is an LLM-powered interface over the entire runtime. It reads event streams from all layers and answers:

"What is happening right now?"
"Why is the negotiation stuck?"
"What happens next in this workflow?"

It supports both streaming and non-streaming responses, with Langfuse tracking on every interaction. When no AI provider is configured, it falls back to a rule-based simulator that summarizes system state from event data.

The Event Bus: Where Everything Connects

All subsystems communicate through typed event streams. The bus has two implementations:

MemoryEventBus: In-memory array, perfect for development and testing
RedisEventBus: Redis Streams with XADD/XREVRANGE, for production persistence

Event types include:

agent.lifecycle — registration, heartbeat, offline transitions
agent.runtime — session lifecycle, thoughts, tool calls
agent.task — A2A message passing between agents
negotiation.events — proposals, counters, accepts, rejections
settlement.events — escrow creation, fund release, ticket minting
wish.events / demand.events / matching.events — workflow pipeline

interface EventEnvelope {
	id: string;
	type: string;
	source: string;
	timestamp: number;
	data: Record<string, unknown>;
	metadata: {
		traceId: string;
		spanId: string;
	};
}

Every event carries trace context, enabling end-to-end flow reconstruction.

What I Learned

Graceful degradation is not optional. The dual-mode runtime (AI + simulated) meant I could develop and demo the entire system without API keys, then flip to real AI when deploying. This pattern should be standard in every agent framework.

Event-driven architecture makes multi-agent systems debuggable. Because every agent action is published to typed event streams, I can reconstruct exactly what happened — even across negotiation rounds, session restarts, and settlement failures.

Human gates are critical for economic execution. Without explicit confirmation before escrow creation and fund release, autonomous agents become a financial liability. The confirmed flag on every settlement method is the simplest correct design.

57 seed agents = instant ecosystem. Seeding the registry with diverse agent cards means every demo starts with a populated marketplace. The auto-seeding pattern (first access triggers population) eliminated initialization order bugs entirely.

The Stack

Runtime: TypeScript, Vercel AI SDK, Zod validation
Events: Redis Streams (prod), in-memory (dev)
Contracts: Solidity, Hardhat, viem
Frontend: Next.js 14, Tailwind CSS
Observability: Langfuse
Infrastructure: Docker Compose, pnpm workspace

Final Thought

WishLive started as a hackathon project. But the architecture — a layered runtime with event-driven communication, dual-mode execution, and on-chain settlement — maps to real production concerns. Agents aren't functions you call. They're actors in a system with state, economics, and coordination.

The code is at github.com/lora-sys/Hackthon.

Building a Full-Stack YouTube Clone with Next.js 15

Tue, 10 Mar 2026 00:00:00 GMT

Introduction

Building a video-sharing platform like YouTube is one of the most challenging yet rewarding projects for full-stack developers. It involves complex requirements: video processing, real-time streaming, user authentication, subscription systems, and much more.

In this technical deep dive, I'll walk you through how I built NewTube - a fully functional YouTube clone using modern web technologies. This isn't just a UI clone; it's a production-ready application with real video processing, AI-powered features, and a scalable architecture.

GitHub Repository: NewTube Clone

Source and setup instructions: NewTube Clone on GitHub. A hosted demo is not currently published.

Project Overview

What is NewTube?

NewTube is a full-featured video-sharing platform that includes:

Video Upload & Streaming: Powered by Mux with HLS adaptive streaming
User Authentication: Complete auth flow with Clerk
Creator Studio: Dashboard for video management and analytics
Subscription System: Follow your favorite creators
Comments & Reactions: Full engagement features
Playlists: Create and manage video collections
AI Features: Auto-generate titles, descriptions, and thumbnails
Responsive Design: Works seamlessly on all devices

Tech Stack at a Glance

Layer	Technology
Frontend	Next.js 15, React 19, TypeScript, Tailwind CSS
Backend	tRPC, Drizzle ORM
Database	PostgreSQL (Neon)
Auth	Clerk
Video	Mux
Storage	UploadThing
Cache	Upstash Redis
Queue	QStash

Architecture Deep Dive

System Architecture

The application follows a three-tier architecture with serverless patterns optimized for modern cloud deployment:

┌─────────────────────────────────────────────────────────────────┐
│                      CLIENT LAYER                                │
│  Browser / Mobile → React Components → tRPC Client              │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│                      API LAYER (tRPC)                            │
│  - protectedProcedure middleware                                 │
│  - Input validation (Zod)                                        │
│  - Rate limiting (Upstash)                                       │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────���───────────────────────────────────────────┐
│                      DATA LAYER                                  │
│  PostgreSQL (Drizzle ORM) ←→ Redis Cache ←→ External Services   │
└─────────────────────────────────────────────────────────────────┘

Project Structure

I organized the codebase using a feature-based module structure (which I call "moubles"):

moubles/
├── videos/
│   ├── server/
│   │   └── procedures.ts    # tRPC API endpoints
│   ├── ui/
│   │   └── components/      # React components
│   └── type.ts              # TypeScript types
├── comments/
├── subscriptions/
├── playlists/
└── ... (other features)

This approach keeps related code together, making the codebase more maintainable as it grows.

Request Flow Example

Here's how a typical API request flows through the system:

// 1. Client Component triggers the query
const [videos] = trpc.videos.getMany.useSuspenseQuery({
	categoryId: selectedCategory
});

// 2. tRPC Client serializes and sends request
// POST /api/trpc/videos.getMany
// Body: { json: { categoryId: "..." } }

// 3. Server-side tRPC handles the request
export const videosRouter = createTRPCRouter({
	getMany: baseProcedure
		.input(
			z.object({
				categoryId: z.string().uuid().optional(),
				limit: z.number().min(1).max(100).default(10)
			})
		)
		.query(async ({ input }) => {
			// Database query with Drizzle ORM
			const data = await db
				.select({
					id: videos.id,
					title: videos.title
					// ... other fields
				})
				.from(videos)
				.where(eq(videos.videoVisiblity, 'public'));

			return { items: data, nextCursor: null };
		})
});

Core Technologies and Features

1. Server Components with SSR Prefetching

Next.js 15's App Router allows us to prefetch data on the server and hydrate it to the client:

// app/(home)/videos/[videoId]/page.tsx (Server Component)
import { HydrateClient, trpc } from '@/trpc/server';

export default async function VideoPage({ params }: { params: { videoId: string } }) {
	// Prefetch on the server
	void trpc.videos.getOne.prefetch({ id: params.videoId });
	void trpc.comments.getMany.prefetch({ videoId: params.videoId });

	return (
		<HydrateClient>
			<VideoSection videoId={params.videoId} />
		</HydrateClient>
	);
}

// VideoSection.tsx (Client Component)
('use client');

export function VideoSection({ videoId }: { videoId: string }) {
	// Data is already prefetched - no loading state!
	const [video] = trpc.videos.getOne.useSuspenseQuery({ id: videoId });

	return <VideoPlayer video={video} />;
}

Benefits:

Zero loading states for initial page load
Better SEO with server-rendered content
Reduced client-side JavaScript

2. Type-Safe API Layer with tRPC

tRPC provides end-to-end type safety without code generation:

// Backend: trpc/router/videos.ts
export const videosRouter = createTRPCRouter({
	create: protectedProcedure.mutation(async ({ ctx }) => {
		const upload = await mux.video.uploads.create({
			new_asset_settings: {
				playback_policies: ['public'],
				inputs: [
					{
						generated_subtitles: [
							{
								language_code: 'en',
								name: 'English'
							}
						]
					}
				]
			},
			cors_origin: process.env.MUX_CORS_ORIGIN || '*'
		});

		const [video] = await db
			.insert(videos)
			.values({
				userId: ctx.user.id,
				title: 'Untitled',
				muxStatus: 'waiting',
				muxUploadId: upload.id
			})
			.returning();

		return { video, url: upload.url };
	})
});

// Frontend: Auto-completion and type checking
const createMutation = trpc.videos.create.useMutation();
//     ^? { video: Video, url: string } - fully typed!

3. Authentication Middleware with Caching

The authentication flow is optimized with a three-tier caching strategy:

// trpc/init.ts
export const protectedProcedure = t.procedure.use(async (opts) => {
	const { ctx } = opts;

	if (!ctx.clerkUserId) {
		throw new TRPCError({ code: 'UNAUTHORIZED' });
	}

	// Tier 1: Check JWT publicMetadata (fastest)
	if (ctx.dbUserId) {
		return opts.next({ ctx: { user: { id: ctx.dbUserId } } });
	}

	// Tier 2: Check Redis cache
	const cachedDbId = await redis.get(`user:dbId:${ctx.clerkUserId}`);
	if (cachedDbId) {
		return opts.next({ ctx: { user: { id: cachedDbId } } });
	}

	// Tier 3: Database query (fallback)
	const [user] = await db.select().from(users).where(eq(users.clerkId, ctx.clerkUserId));

	// Cache for future requests
	await redis.set(`user:dbId:${ctx.clerkUserId}`, user.id, { ex: 3600 });

	// Update JWT for next time
	await clerkClient.users.updateUserMetadata(ctx.clerkUserId, {
		publicMetadata: { dbUserId: user.id }
	});

	return opts.next({ ctx: { user } });
});

4. Video Processing Pipeline

The video upload and processing flow uses webhooks for async updates:

User Upload → Mux Upload URL → Mux Processing → Webhook → Database Update
     │              │                │               │            │
     └──────────────┴────────────────┴───────────────┴────────────┘
                    Asynchronous flow with real-time updates

// app/api/videos/webhook/route.ts
export async function POST(req: Request) {
	const payload = await req.json();

	switch (payload.type) {
		case 'video.asset.ready': {
			const asset = payload.data;
			const playbackId = asset.playback_ids[0].id;

			// Generate thumbnail from Mux
			const thumbnailUrl = `https://image.mux.com/${playbackId}/thumbnail.jpg`;
			const uploaded = await utapi.uploadFilesFromUrl(thumbnailUrl);

			// Update database
			await db
				.update(videos)
				.set({
					muxPlaybackId: playbackId,
					muxStatus: 'ready',
					duration: Math.round(asset.duration),
					thumbnailUrl: uploaded.data.ufsUrl
				})
				.where(eq(videos.muxUploadId, asset.upload_id));

			break;
		}
	}

	return new Response('OK');
}

5. AI-Powered Content Generation

Using QStash for async AI tasks:

// Trigger AI task
const { workflowRunId } = await workflow.trigger({
	url: `${process.env.QSTASH_WORKFLOW_URL}/api/videos/workflows/title`,
	body: { userId, videoId }
});

// Workflow handler
export async function POST(req: Request) {
	const { videoId } = await req.json();

	// Get video transcript from Mux
	const track = await mux.video.tracks.get(video.muxTrackId);
	const transcript = track.text;

	// Generate title with AI
	const response = await openai.chat.completions.create({
		model: 'gpt-4',
		messages: [
			{
				role: 'user',
				content: `Generate a YouTube title based on: ${transcript}`
			}
		]
	});

	const title = response.choices[0].message.content;

	// Update database
	await db.update(videos).set({ title }).where(eq(videos.id, videoId));
}

6. Optimistic Updates for Better UX

const utils = trpc.useUtils();

const likeMutation = trpc.videoReactions.like.useMutation({
	onMutate: async ({ videoId }) => {
		// Cancel outgoing refetches
		await utils.videos.getOne.cancel({ id: videoId });

		// Snapshot previous value
		const previous = utils.videos.getOne.getData({ id: videoId });

		// Optimistically update UI
		utils.videos.getOne.setData({ id: videoId }, (old) => ({
			...old!,
			viewerReaction: 'like',
			likeCount: old!.likeCount + 1
		}));

		return { previous };
	},
	onError: (err, { videoId }, context) => {
		// Rollback on error
		utils.videos.getOne.setData({ id: videoId }, context.previous);
		toast.error('Failed to like video');
	},
	onSettled: ({ videoId }) => {
		// Refetch to ensure consistency
		utils.videos.getOne.invalidate({ id: videoId });
		utils.playlists.getLiked.invalidate();
	}
});

Challenges and Solutions

Challenge 1: SSR Authentication with Protected Routes

Problem: When using SSR with protected routes, the server doesn't have access to the client's authentication state, causing 401 errors or hydration mismatches.

Solution: Implement a multi-layered approach:

// In protected components
"use client";

import { useAuth, useClerk } from "@clerk/nextjs";
import { useEffect, useRef } from "react";

export function ProtectedSection() {
  const { isSignedIn, isLoaded } = useAuth();
  const { openSignIn } = useClerk();
  const hasTriggeredSignIn = useRef(false);

  useEffect(() => {
    // Prevent multiple calls with useRef
    if (isLoaded && !isSignedIn && !hasTriggeredSignIn.current) {
      hasTriggeredSignIn.current = true;
      openSignIn();
    }
  }, [isLoaded, isSignedIn, openSignIn]);

  if (!isLoaded) {
    return <Skeleton />;
  }

  if (!isSignedIn) {
    return null;
  }

  return <ActualContent />;
}

Key Insight: Using useRef prevents the sign-in modal from opening multiple times during React's strict mode double-render.

Challenge 2: Hydration Mismatch in Mobile Detection

Problem: A mobile detection hook caused hydration errors because the initial value differed between server and client.

// ❌ Problem: undefined on server, boolean on client
const [isMobile, setIsMobile] = useState<boolean | undefined>(undefined);
return !!isMobile;

Solution: Default to false and only update on client:

// ✅ Fixed: Consistent initial value
const [isMobile, setIsMobile] = useState<boolean>(false);

useEffect(() => {
	const checkMobile = () => {
		setIsMobile(window.innerWidth < 768);
	};

	checkMobile();
	window.addEventListener('resize', checkMobile);
	return () => window.removeEventListener('resize', checkMobile);
}, []);

return isMobile;

Challenge 3: Cache Invalidation Across Features

Problem: When a user likes a video, the "Liked Videos" playlist wasn't updating in real-time.

Solution: Invalidate all related queries:

const likeMutation = trpc.videoReactions.like.useMutation({
	onSuccess: () => {
		// Primary data
		utils.videos.getOne.invalidate({ id: videoId });

		// Related features
		utils.playlists.getLiked.invalidate();
		utils.playlists.getLikedPreview.invalidate();
	}
});

Challenge 4: CORS Configuration for Video Uploads

Problem: In production, Mux uploads failed due to CORS restrictions.

Solution: Use environment-based CORS configuration:

const upload = await mux.video.uploads.create({
	new_asset_settings: {
		/* ... */
	},
	// Development: "*" | Production: specific domain
	cors_origin: process.env.MUX_CORS_ORIGIN || '*'
});

Challenge 5: Real-time Video Processing Status

Problem: Users couldn't see when their video finished processing.

Solution: Implement polling with automatic cleanup:

const [workflowRunId, setWorkflowRunId] = useState<string | null>(null);

useEffect(() => {
	if (!workflowRunId) return;

	const interval = setInterval(() => {
		// Invalidate cache to refetch latest data
		utils.studio.getOne.invalidate({ id: videoId });
	}, 2000);

	// Cleanup after 60 seconds
	const timeout = setTimeout(() => {
		clearInterval(interval);
		toast.info('Processing may take longer than expected');
	}, 60000);

	return () => {
		clearInterval(interval);
		clearTimeout(timeout);
	};
}, [workflowRunId, videoId, utils]);

Conclusion

Building NewTube was an incredible learning experience that pushed me to solve real-world problems at scale. Here's what I learned:

Key Takeaways

Type Safety is Non-Negotiable: tRPC + TypeScript + Zod eliminated entire categories of bugs
Server Components are Game-Changers: SSR prefetching dramatically improved UX and SEO
Caching Strategy Matters: The three-tier auth caching reduced database queries by 90%
Webhooks Enable Async Workflows: Critical for video processing and AI tasks
Feature-Based Architecture Scales: The "moubles" structure kept the codebase maintainable

What's Next?

Future improvements I'm planning:

Real-time notifications with Server-Sent Events
Video chapters and timestamps
Live streaming support
Advanced analytics dashboard
Mobile app with React Native

Resources

GitHub Repository: NewTube Clone
Next.js Documentation: nextjs.org/docs
tRPC Documentation: trpc.io/docs
Drizzle ORM: orm.drizzle.team

Thank you for reading! If you found this article helpful, please give the repository a ⭐ and feel free to reach out with questions.

Happy coding! 🚀

Lora

AI Engineering Harness: 从 Vibe Coding 到工程化

背景：Vibe 之后，正式接管

实际场景

问题：AI 工程的四大痛点

传统方案的代价

解决方案：AI Engineering Harness

核心理念

四个关键特性

1️⃣ 证据驱动的工程纪律

2️⃣ 18 类专业化 Agent

3️⃣ 工作流隔离（Worktree）

4️⃣ 上下文分层加载（L0-L3）

核心架构：18 类 Agent 的协作

架构总览

Agent 角色详解

🧠 Coordinator（协调者）

🎨 Frontend Agent（前端工程师）

⚙️ Backend Agent（后端工程师）

🐛 Bug Hunter（缺陷猎人）

👁️ Behavior Reviewer（行为审查员）

闭环流程：从 Issue 到 Merge

完整流程图

9 个 Phase 详解

Phase 0: PRD → Coordinator

Phase 1: Planning → Coordinator

Phase 2: Worktree → Owner Agent

Phase 3: Implementation → Owner Agent

Phase 4: CI Gate → Coordinator

Phase 5: Review → Bug Hunter + Behavior Reviewer

Phase 6: Evidence → Evidence Aggregator

Phase 7: Merge → Coordinator

Phase 8: Memory → Coordinator

关键技术：证据闸门

什么是证据闸门？

证据目录结构

证据检查清单

实战：接管一个失控的仓库

场景

Step 1: 安装（5 分钟）

Step 2: Quick Scan（2 分钟）

Step 3: 接管（10 分钟）

Step 4: 推一个 Issue 到 Done（10 分钟）

Step 5: 验证（3 分钟）

安装与配置

一行安装

精确安装

兼容 40 个 CLI Agent

进阶用法

跨 CLI 接力

并行开发

CI 自愈

30 秒拉起新项目

效果展示

数据对比

真实案例：install-session-hook

9 条运行原则

总结

核心价值

适用场景

快速开始

开源与社区

Free Vision Skill: 给文本模型装上一双按需调用的眼睛

背景：为什么文本模型需要"眼睛"？

实际场景

问题：传统方案的四大痛点

举个实际例子

解决方案：Free Vision Skill

核心理念

工作流程

关键特性

核心技术：VEP/1 协议

VEP 格式

字段说明

Token 对比

架构设计

系统架构

核心模块

1. Provider System (src/providers.ts)

2. 智能缓存 (src/cache.ts)

1. Provider System (`src/providers.ts`)

2. 智能缓存 (`src/cache.ts`)

3. 并发控制 (`src/pool.ts`)

4. VEP 生成 (`src/vep.ts`)

第二步：编写插件清单 `plugin.yaml`