GPU_GUARD_MONOREPO/docs/superpowers/plans/2026-05-04-netabrowser-cli-s1-plan.md
2026-05-20 21:39:12 +08:00

3174 lines
108 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# netabrowser-cli S1 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
>
> **上位文档**[`../specs/2026-05-04-netabrowser-cli-s1-design.md`](../specs/2026-05-04-netabrowser-cli-s1-design.md)
> **Git 策略**:用户要求"全部完成 + 联调通过后统一提交"。每 task 末尾标注 `Checkpoint`(仅作进度标记,不做 git commit
**Goal:** 在 Neta monorepo 内交付 netabrowser-cli S1 基础设施BrowserDaemonService嵌入 backend+ netabrowser-cli 二进制 + skill 元数据,提供反风控+拟人化的浏览器自动化能力,统一服务于 NetaClaw Agent 探索期CLI和后端业务固化期@Inject service
**Architecture:** Service-First 架构:核心是 backend 内的 `BrowserDaemonService`Midway @Singleton)持有 `Map<sessionName, BrowserContext>`CLI 是 thin client 通过 HTTP loopback + secret 调 backend底层用 `patchright` (npm) 启动 vendored `neta-chromium` (fingerprint-chromium);拟人化用 `ghost-cursor` + 自加套,分 full/fast/off 三档session-name 串行化锁LRU 软上限调度。
**Tech Stack:** Midway.js 3.20 / TypeScript 5.9 / `patchright` (npm) / `ghost-cursor` (npm) / fingerprint-chromium 144.0.7559.132 (vendored 二进制) / commander (CLI 解析) / jest+ts-jest / Inno Setup / yao-pkg
---
## ⚠️ v2 修复总览(架构 review 后必读)
本 plan 经过架构师交叉验证发现 8 项 P0/P1 问题,已就地修复。实施时优先理解这些修复点:
| # | 问题 | 修复落点 |
|---|---|---|
| **P0-1** | Controller 路径与 Cool Admin 风格冲突 | Task 11 全部 controller 改用 `@Provide()` + `@Controller('/admin/browser-daemon/<sub>')` + 方法相对路径,参考 `netaclaw/controller/admin/agent_channel.ts` 风格 |
| **P0-2** | dev 模式 dataDir = `<cwd>/dist`build 清空状态 | 新增 Task 4b`resolveBrowserDataDir()` 独立解析dev 模式 → `<monorepo>/.netabrowser-data/`prod → `<dataDir>/.netabrowser/` |
| **P0-3** | browser-daemon 模块挂载错位 | Task 10 Step 5 明确:把 `BrowserControlAuthMiddleware` 加到 `netaclaw/config.ts``middlewares` 数组,依靠 `match()` 仅作用于 `/admin/browser-daemon/*` 路径 |
| **P0-4** | ghost-cursor 类型与 patchright 不兼容 | Task 8 Step 3 humanizer 内部 `createCursor(page as any)` cast |
| **P1-5** | dev 模式 NETA_TRAY_SECRET 未设 → 全 401 | Task 10 middleware 加 dev 旁路:`process.env.NODE_ENV !== 'production' && expected === ''` 时跳过 secret 校验 |
| **P1-6** | CLI runtime-info 路径错位cwd 不对) | Task 14 候选改为 `<monorepo-root>/packages/backend/dist/runtime-info.json` + 强制 `NETA_RUNTIME_INFO` 环境变量优先 |
| **P1-7** | @Init/@Destroy lifecycle + 60min idle 自动回收缺失 | 新增 Task 9b`runtime/cleanup.ts` + daemon.service `@Init`/`@Destroy` + scheduler idle timer |
| **P1-8** | CLI 与会话级 mode 协同 bug | service 移除 modeMapservice-statelessCLI 端用环境变量 `NETA_BROWSER_HUMANIZE_MODE` 作"会话级默认",命令级 `--mode` 覆盖 |
**额外路径修正**Task 4 `chromium-launcher` `__dirname` 上溯 **7 层**(不是 6 层)—— 实际路径 `dist/modules/netaclaw/browser-daemon/runtime/` 到 monorepo root 是 7 个 `..`
**版本锁定**Task 1 安装命令改为 `pnpm add patchright@1.59.4 ghost-cursor@1.4.2`(已验证兼容版本)。
---
## 文件结构
### 后端模块(新增 14 个文件)
| 路径 | 职责 |
|---|---|
| `packages/backend/src/modules/netaclaw/browser-daemon/config.ts` | 模块配置 + `maxActiveSessions` 等 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` | ★ BrowserDaemonService 核心,统一锁 + 调度入口 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts` | 拟人化包装ghost-cursor + 三档) |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts` | neta-chromium 指纹参数生成 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts` | AI ref 协议spike #1 选定方案) |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts` | 路径解析 + patchright launch 包装 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts` | Map<name, context> + 串行化锁 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` | LRU 软上限调度 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts` | @Destroy 时优雅关闭 |
| `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts` | loopback + secret auth |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts` | /open /close /list /stats |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts` | /click /fill /type /scroll /hover /press |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts` | /goto /back /forward /reload |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts` | /cookie-list /cookie-set /state-save /state-load |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts` | /snapshot /screenshot /eval /run-code |
### netabrowser-cli 包(新增 13 个文件)
| 路径 | 职责 |
|---|---|
| `packages/netabrowser-cli/package.json` | bin 声明 + patchright/commander 等依赖 |
| `packages/netabrowser-cli/tsconfig.json` | TS 配置 |
| `packages/netabrowser-cli/src/bin/main.ts` | CLI 入口commander 解析) |
| `packages/netabrowser-cli/src/client/runtime-info.ts` | 读 backend runtime-info.json |
| `packages/netabrowser-cli/src/client/http-client.ts` | HTTP 请求 + secret header |
| `packages/netabrowser-cli/src/output/formatter.ts` | --raw vs 默认 + 错误格式化 |
| `packages/netabrowser-cli/src/commands/session.ts` | open/close/list共用一个文件 |
| `packages/netabrowser-cli/src/commands/interaction.ts` | click/fill/type/scroll/hover/press |
| `packages/netabrowser-cli/src/commands/navigation.ts` | goto/back/forward/reload |
| `packages/netabrowser-cli/src/commands/state.ts` | cookie / state |
| `packages/netabrowser-cli/src/commands/inspect.ts` | snapshot/screenshot/eval/run-code |
### 测试文件(新增 11 个)
| 路径 | 测试对象 |
|---|---|
| `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts` | 路径解析 + launch args 拼装 |
| `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts` | seed → args 派生 |
| `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts` | 三档行为 + 注入 random source 验证延迟范围 |
| `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts` | 串行化锁竞态 |
| `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts` | LRU + 软上限 + 队列 |
| `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts` | ref 协议契约 |
| `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts` | 编排open/click/cookie/close mock 全链路 |
| `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts` | loopback + secret 校验 |
| `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts` | HTTP ↔ service 等价性 contract |
| `packages/netabrowser-cli/tests/cli-parse.test.ts` | commander 参数解析 |
| `packages/netabrowser-cli/tests/output-formatter.test.ts` | 输出格式 |
### Skill 元数据(新增 5 个文件)
| 路径 | 职责 |
|---|---|
| `packages/backend/skills/netabrowser-cli/SKILL.md` | 给 NetaClaw Agent 看的命令清单 |
| `packages/backend/skills/netabrowser-cli/references/humanization.md` | 三档详解 |
| `packages/backend/skills/netabrowser-cli/references/fingerprint.md` | 指纹参数 |
| `packages/backend/skills/netabrowser-cli/references/proxy.md` | 代理配置 |
| `packages/backend/skills/netabrowser-cli/references/examples.md` | 典型场景 |
### 集成改动
| 路径 | 改动 |
|---|---|
| `pnpm-workspace.yaml` | 新增 `packages/netabrowser-cli` |
| `packages/backend/package.json` | 新增 `patchright` + `ghost-cursor` 依赖 |
| `packages/backend/installer/setup.iss` | `[Files]` 段加 chromium/win64 + netabrowser-cli.exe |
| `packages/backend/scripts/build-windows-installer.js` | 复制 chromium 二进制步骤 |
| `packages/backend/scripts/pkg-build.js` | 增加 netabrowser-cli pkg 编译 |
---
## 实施依赖图
```
Phase 0: T0(包脚手架) → T1(npm 依赖)
Phase 1: T2(Spike #1 ref 协议) ‖ T3(Spike #2 ghost-cursor 兼容) ← 阻塞门
Phase 2: T4(launcher) → T5(registry+lock) → T6(scheduler) → T7(open/close)
T8(humanizer + click/fill) → T9(state/cookie/snapshot)
Phase 3: T10(auth middleware) → T11(controllers) → T12(contract test)
Phase 4: T13(cli bin) → T14(client) → T15-17(命令实现) → T18(formatter)
Phase 5: T19-20(skill 元数据)
Phase 6: T21(pkg build) → T22(installer) → T23(路径解析)
Phase 7: T24(联调冒烟) → T25(性能验证) → T26(文档)
```
---
## Task 0netabrowser-cli 包脚手架
**Files:**
- Create: `packages/netabrowser-cli/package.json`
- Create: `packages/netabrowser-cli/tsconfig.json`
- Modify: `pnpm-workspace.yaml`
- [ ] **Step 1: 把 netabrowser-cli 加入 pnpm workspace**
`pnpm-workspace.yaml`,确认含有 `packages/*` 通配符(应该已有)。如无 `packages/*` 通配符,加上:
```yaml
packages:
- 'packages/*'
```
无需修改的话跳过。
- [ ] **Step 2: 创建 netabrowser-cli/package.json**
```json
{
"name": "@neta/netabrowser-cli",
"version": "0.1.0",
"description": "Neta 反风控+拟人化浏览器自动化 CLI",
"private": true,
"type": "module",
"bin": {
"netabrowser-cli": "./dist/bin/main.js"
},
"scripts": {
"build": "tsc -p tsconfig.json",
"test": "jest"
},
"dependencies": {
"commander": "^12.0.0",
"axios": "^1.7.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"jest": "^29.7.0",
"ts-jest": "^29.1.5",
"typescript": "^5.9.0"
}
}
```
- [ ] **Step 3: 创建 netabrowser-cli/tsconfig.json**
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"resolveJsonModule": true,
"declaration": false
},
"include": ["src/**/*"]
}
```
- [ ] **Step 4: 验证 pnpm 识别 workspace**
```bash
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
pnpm install
pnpm list -r --depth=-1 | grep netabrowser-cli
```
预期:列表中出现 `@neta/netabrowser-cli`
- [ ] **Step 5: Checkpoint**
`Task 0 完成`:包注册到 workspace可以独立 build/test。
---
## Task 1backend 安装 patchright + ghost-cursor 依赖
**Files:**
- Modify: `packages/backend/package.json`
- [ ] **Step 1: 在 backend 安装 patchrightnpm 包,锁定版本)**
```bash
cd packages/backend
pnpm add patchright@1.59.4
```
> 锁定 1.59.4:经 review 验证此版本兼容 Node 22 + Midway 3.20 + neta-chromium 144.0.7559.132。后续升级前需重测兼容性。
- [ ] **Step 2: 安装 ghost-cursor锁定版本**
```bash
pnpm add ghost-cursor@1.4.2
```
- [ ] **Step 3: 验证 patchright 能 import**
跑一段临时脚本:
```bash
cd packages/backend
node -e "import('patchright').then(p => console.log('patchright loaded:', Object.keys(p).join(',')))"
```
预期输出含 `chromium,firefox,webkit,...`
- [ ] **Step 4: 验证 ghost-cursor 能 import**
```bash
node -e "import('ghost-cursor').then(p => console.log('ghost-cursor loaded:', Object.keys(p).join(',')))"
```
预期含 `createCursor,path,...`
- [ ] **Step 5: Checkpoint**
`Task 1 完成`:依赖装好,可以在 service 代码 import 使用。
---
## Task 2 [SPIKE #1]AI Ref 协议方案验证
> 这是 spec §11 的阻塞门 spike。**必须先做,否则后续 interaction 命令无法实现**。
**目标**:选定一种 AI ref 协议方案并 demo 跑通。
候选方案:
- **A. 自实现**snapshot 时给 DOM 注入 `data-ai-ref="e15"` 属性 + 内存维护 `Map<sessionName, Map<ref, ElementHandle>>`
- **B. 改用 selector**:命令直接接收 selector 字符串(`click 'button:has-text("登录")'`),不要 ref
- **C. vendor playwright-cli ref 代码**:剥离 playwright-cli 内部 ref 实现引入 netabrowser-cli
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts`
- [ ] **Step 1: 写 spike 测试(方案 A先验证最简实现**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts
import { chromium } from 'patchright';
import * as path from 'node:path';
import * as fs from 'node:fs';
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
describe('Spike #1: AI ref 协议(方案 A 自实现)', () => {
jest.setTimeout(60000);
it('A: snapshot 注入 data-ai-ref可基于 ref 定位元素', async () => {
if (!fs.existsSync(CHROME)) {
console.warn('SKIP: neta-chromium not found at', CHROME);
return;
}
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-1');
const ctx = await chromium.launchPersistentContext(tmpProfile, {
executablePath: CHROME,
headless: true,
args: ['--fingerprint=12345'],
});
const page = ctx.pages()[0] || (await ctx.newPage());
await page.goto('data:text/html,<button id=login>Login</button><a href=#>Link</a>');
// 1. snapshot给所有可交互元素注入 data-ai-ref
const refs = await page.evaluate(() => {
const interactive = Array.from(document.querySelectorAll('button, a, input, select, textarea'));
const out: { ref: string; tag: string; text: string }[] = [];
interactive.forEach((el, i) => {
const ref = `e${i + 1}`;
el.setAttribute('data-ai-ref', ref);
out.push({ ref, tag: el.tagName.toLowerCase(), text: (el.textContent || '').trim() });
});
return out;
});
expect(refs).toEqual([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'a', text: 'Link' },
]);
// 2. 通过 ref click
await page.locator('[data-ai-ref="e1"]').click();
// 不报错 = 通过
await ctx.close();
fs.rmSync(tmpProfile, { recursive: true, force: true });
});
});
```
- [ ] **Step 2: 跑 spike 测试**
```bash
cd packages/backend
pnpm jest test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts -t 'A:'
```
预期PASS含 fingerprint chromium 已在 vendor 里)
- [ ] **Step 3: 评审 spike 结果**
如果方案 A PASS写入决策文件
```bash
cat > packages/backend/src/modules/netaclaw/browser-daemon/SPIKE_DECISION.md << 'EOF'
# Spike #1 决策AI Ref 协议
**选定方案 A自实现 data-ai-ref 注入**
## 理由
- 实现简单(一段 page.evaluate JS 即可)
- 不引入额外依赖
- 完全可控
## 实现要点
1. snapshot 命令时调用 `injectRefs(page)`,给所有可交互元素加 `data-ai-ref="eN"`
2. service 内存维护 `Map<sessionName, Map<ref, locator-info>>` 用于审计/调试
3. interaction 命令收到 ref 后用 `page.locator('[data-ai-ref="${ref}"]')` 定位
4. 页面跳转/重新 snapshot 后旧 ref 失效DOM 变化)
## 不选 B/C 的理由
- BselectorAI 写 selector 容易错ref 抽象更友好
- Cvendor playwright-cliplaywright-cli 用 socket+state 同步,移植成本高
EOF
```
如果方案 A FAIL极不可能降级 B/C 重写 spike。
- [ ] **Step 4: Checkpoint**
`Task 2 完成`:方案 A 通过,决策记录在 `SPIKE_DECISION.md`。后续 snapshot-ref.service.ts 实现该方案。
---
## Task 3 [SPIKE #2]ghost-cursor on patchright 兼容性验证
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts`
- [ ] **Step 1: 写 spike 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
import { chromium } from 'patchright';
import { createCursor } from 'ghost-cursor';
import * as path from 'node:path';
import * as fs from 'node:fs';
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
describe('Spike #2: ghost-cursor on patchright + neta-chromium', () => {
jest.setTimeout(60000);
it('cursor.click 能在 patchright 启动的 neta-chromium 上工作', async () => {
if (!fs.existsSync(CHROME)) {
console.warn('SKIP: neta-chromium not found');
return;
}
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-2');
const ctx = await chromium.launchPersistentContext(tmpProfile, {
executablePath: CHROME,
headless: true,
args: ['--fingerprint=99999'],
});
const page = ctx.pages()[0] || (await ctx.newPage());
await page.goto('data:text/html,<button id=b style="position:absolute;top:200px;left:300px">Click me</button><script>document.getElementById("b").addEventListener("click",()=>document.title="CLICKED")</script>');
// 创建 ghost cursor
const cursor = createCursor(page as any);
// 拿目标坐标
const box = await page.locator('#b').boundingBox();
expect(box).not.toBeNull();
// 用 ghost-cursor 移动并点击(贝塞尔轨迹)
await cursor.moveTo({ x: box!.x + box!.width / 2, y: box!.y + box!.height / 2 });
await page.mouse.down();
await page.waitForTimeout(80);
await page.mouse.up();
// 等待 click 生效
await page.waitForFunction(() => document.title === 'CLICKED', null, { timeout: 5000 });
expect(await page.title()).toBe('CLICKED');
// 验证 navigator.webdriver 仍为 falsepatchright 反检测有效)
const wd = await page.evaluate(() => navigator.webdriver);
expect(wd).toBe(false);
await ctx.close();
fs.rmSync(tmpProfile, { recursive: true, force: true });
});
});
```
- [ ] **Step 2: 跑 spike**
```bash
pnpm jest test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
```
预期PASS含 navigator.webdriver === false
- [ ] **Step 3: 写决策**
如果通过append `SPIKE_DECISION.md`
```markdown
# Spike #2 决策ghost-cursor 兼容
✅ ghost-cursor 在 patchright + neta-chromium 上工作正常。
- cursor.moveTo 走贝塞尔轨迹patchright 没拦截 mouse 事件
- click 后 navigator.webdriver 仍 falsepatchright 反检测维持)
humanizer.service.ts 直接 `import { createCursor } from 'ghost-cursor'` 即可。
```
如果失败:选用纯自写贝塞尔(替代方案,备选脚本另准备)。
- [ ] **Step 4: Checkpoint**
`Task 3 完成`ghost-cursor 兼容性确认,可投入 humanizer 实现。
---
## Task 4chromium-launcher路径解析 + launch args 拼装)
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts`
- [ ] **Step 1: 写 fingerprint 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts
import { FingerprintService } from '../../../../src/modules/netaclaw/browser-daemon/service/fingerprint.service.js';
describe('FingerprintService', () => {
const svc = new FingerprintService();
it('fromSeed: 仅 seed 生成完整 args', () => {
const args = svc.fromSeed(12345);
expect(args).toEqual(expect.arrayContaining([
'--fingerprint=12345',
'--fingerprint-platform=Windows',
'--fingerprint-platform-version=10.0.0',
'--fingerprint-brand=Chrome',
'--fingerprint-hardware-concurrency=8',
'--fingerprint-language=zh-CN',
]));
});
it('merge: 指定字段覆盖默认', () => {
const args = svc.merge({ seed: 1, language: 'en-US', hardwareConcurrency: 16 });
expect(args).toContain('--fingerprint=1');
expect(args).toContain('--fingerprint-language=en-US');
expect(args).toContain('--fingerprint-hardware-concurrency=16');
});
it('seed 不同生成不同 fingerprint arg', () => {
expect(svc.fromSeed(1)).toContain('--fingerprint=1');
expect(svc.fromSeed(2)).toContain('--fingerprint=2');
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
cd packages/backend && pnpm jest fingerprint.service
```
预期FAILservice 不存在)
- [ ] **Step 3: 实现 FingerprintService**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface FingerprintParams {
seed?: number;
platform?: 'Windows' | 'macOS' | 'Linux';
platformVersion?: string;
brand?: string;
brandVersion?: string;
hardwareConcurrency?: number;
language?: string;
timezone?: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class FingerprintService {
/** 仅 seed → 全套默认 args */
fromSeed(seed: number): string[] {
return this.merge({ seed });
}
/** 自定义参数 + 默认值 → args */
merge(p: FingerprintParams): string[] {
const args: string[] = [];
if (p.seed != null) args.push(`--fingerprint=${p.seed}`);
args.push(`--fingerprint-platform=${p.platform ?? 'Windows'}`);
args.push(`--fingerprint-platform-version=${p.platformVersion ?? '10.0.0'}`);
args.push(`--fingerprint-brand=${p.brand ?? 'Chrome'}`);
if (p.brandVersion) args.push(`--fingerprint-brand-version=${p.brandVersion}`);
args.push(`--fingerprint-hardware-concurrency=${p.hardwareConcurrency ?? 8}`);
args.push(`--fingerprint-language=${p.language ?? 'zh-CN'}`);
if (p.timezone) args.push(`--fingerprint-timezone=${p.timezone}`);
return args;
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest fingerprint.service
```
预期PASS3 tests
- [ ] **Step 5: 写 chromium-launcher 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts
import { resolveChromiumPath, buildLaunchArgs } from '../../../../src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.js';
import * as path from 'node:path';
describe('chromium-launcher', () => {
describe('resolveChromiumPath', () => {
afterEach(() => { delete process.env.NETA_CHROMIUM_PATH; });
it('优先环境变量', () => {
process.env.NETA_CHROMIUM_PATH = '/custom/chrome.exe';
expect(resolveChromiumPath({ isPkg: false })).toBe('/custom/chrome.exe');
});
it('pkg 模式execPath 同目录 chromium/win64/chrome.exe', () => {
const result = resolveChromiumPath({ isPkg: true, execPath: 'C:/Program Files/Neta/backend.exe' });
expect(result).toMatch(/Program Files\/Neta\/chromium\/win64\/chrome\.exe$/);
});
it('dev 模式monorepo packages/netabrowser-cli/chromium/win64/chrome.exe', () => {
const result = resolveChromiumPath({ isPkg: false });
expect(result).toMatch(/netabrowser-cli\/chromium\/win64\/chrome\.exe$/);
});
});
describe('buildLaunchArgs', () => {
it('合并指纹 args + 用户自定义 args', () => {
const args = buildLaunchArgs({
fingerprintArgs: ['--fingerprint=42', '--fingerprint-language=zh-CN'],
extraArgs: ['--disable-popup-blocking'],
});
expect(args).toEqual(expect.arrayContaining([
'--fingerprint=42',
'--fingerprint-language=zh-CN',
'--disable-popup-blocking',
]));
});
});
});
```
- [ ] **Step 6: 实现 chromium-launcher**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts
import * as path from 'node:path';
export interface ResolveOpts {
isPkg?: boolean;
execPath?: string;
}
/** 解析 neta-chromium 可执行文件路径 */
export function resolveChromiumPath(opts: ResolveOpts = {}): string {
if (process.env.NETA_CHROMIUM_PATH) return process.env.NETA_CHROMIUM_PATH;
const isPkg = opts.isPkg ?? !!(process as any).pkg;
if (isPkg) {
const execPath = opts.execPath ?? process.execPath;
return path.join(path.dirname(execPath), 'chromium', 'win64', 'chrome.exe').replace(/\\/g, '/');
}
// dev 模式:从 backend src 出发,定位 monorepo 内 packages/netabrowser-cli/chromium/win64/chrome.exe
// 编译后 __dirname = packages/backend/dist/modules/netaclaw/browser-daemon/runtime
// 路径计算runtime → browser-daemon → netaclaw → modules → dist → backend → packages → root7 层 ..
// 然后 root/packages/netabrowser-cli/chromium/win64/chrome.exe
return path.resolve(__dirname, '../../../../../../../packages/netabrowser-cli/chromium/win64/chrome.exe').replace(/\\/g, '/');
}
export interface LaunchArgsOpts {
fingerprintArgs: string[];
extraArgs?: string[];
}
export function buildLaunchArgs(opts: LaunchArgsOpts): string[] {
return [...opts.fingerprintArgs, ...(opts.extraArgs ?? [])];
}
```
- [ ] **Step 7: 跑 chromium-launcher 测试**
```bash
pnpm jest chromium-launcher
```
预期PASS4 tests
- [ ] **Step 8: Checkpoint**
`Task 4 完成`:路径解析 + 指纹参数生成 + launch args 拼装可用。
---
## Task 4bBrowserDataDir 独立解析(**新增 P0 修复**
**问题**`comm/data-dir.ts:resolveDataDir()` 在 dev 模式 fallback 到 `<cwd>/dist`。把 `browser-profiles/` `states/` 放 dist 下,每次 `pnpm build` 会清空,状态全丢。
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts
import { resolveBrowserDataDir, getProfileDir, getStateDir } from '../../../../src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.js';
import * as path from 'node:path';
describe('BrowserDataDir', () => {
afterEach(() => {
delete process.env.NETA_BROWSER_DATA_DIR;
delete process.env.NETA_DATA_DIR;
});
it('优先 NETA_BROWSER_DATA_DIR 环境变量', () => {
process.env.NETA_BROWSER_DATA_DIR = '/custom/browser';
expect(resolveBrowserDataDir({ isPkg: false })).toBe('/custom/browser');
});
it('pkg 模式dataDir/.netabrowser', () => {
const r = resolveBrowserDataDir({ isPkg: true, execDir: 'C:/Program Files/Neta' });
expect(r.replace(/\\/g, '/')).toBe('C:/Program Files/Neta/data/.netabrowser');
});
it('dev 模式monorepo/.netabrowser-data不复用 dist', () => {
const r = resolveBrowserDataDir({ isPkg: false, monorepoRoot: '/repo' });
expect(r.replace(/\\/g, '/')).toBe('/repo/.netabrowser-data');
});
it('getProfileDir / getStateDir 返回子路径', () => {
process.env.NETA_BROWSER_DATA_DIR = '/x';
expect(getProfileDir('s1').replace(/\\/g, '/')).toBe('/x/profiles/s1');
expect(getStateDir('s1').replace(/\\/g, '/')).toBe('/x/states/s1.json');
});
});
```
- [ ] **Step 2: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts
import * as path from 'node:path';
import { resolveDataDir } from '../../../../comm/data-dir.js';
export interface ResolveBrowserDataDirOpts {
isPkg?: boolean;
execDir?: string;
monorepoRoot?: string;
}
/**
* netabrowser-cli 自己的状态根目录,独立于 backend dist/。
* - prod: <dataDir>/.netabrowser/
* - dev: <monorepo-root>/.netabrowser-data/ (不放 dist 下,防 build 清空)
*/
export function resolveBrowserDataDir(opts: ResolveBrowserDataDirOpts = {}): string {
if (process.env.NETA_BROWSER_DATA_DIR) return path.resolve(process.env.NETA_BROWSER_DATA_DIR);
const isPkg = opts.isPkg ?? !!(process as any).pkg;
if (isPkg) {
const dataDir = resolveDataDir({ isPkg: true, execDir: opts.execDir });
return path.join(dataDir, '.netabrowser');
}
// dev: 从 backend 出发上溯到 monorepo root
const monorepoRoot = opts.monorepoRoot
?? path.resolve(__dirname, '../../../../../../../');
return path.join(monorepoRoot, '.netabrowser-data');
}
export function getProfileDir(sessionName: string): string {
return path.join(resolveBrowserDataDir(), 'profiles', sessionName);
}
export function getStateDir(sessionName: string): string {
return path.join(resolveBrowserDataDir(), 'states', `${sessionName}.json`);
}
```
- [ ] **Step 3: 跑测试看通过**
```bash
pnpm jest browser-data-dir
```
预期PASS4 tests
- [ ] **Step 4: 修改 Task 7 daemon.service 中的 import**
将 daemon.service.ts 的:
```typescript
import { resolveDataDir } from '../../../../comm/data-dir.js';
// ...
const profileRoot = path.join(resolveDataDir(), 'browser-profiles');
const profilePath = path.join(profileRoot, opts.profileDir ?? opts.sessionName);
```
改为:
```typescript
import { getProfileDir } from '../runtime/browser-data-dir.js';
// ...
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
```
同样将 saveState/loadState 用到的状态文件路径改为 `getStateDir(sessionName)`
- [ ] **Step 5: 更新 .gitignore**
`.netabrowser-data/` 加入 monorepo 根 `.gitignore`
```bash
echo ".netabrowser-data/" >> C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo/.gitignore
```
- [ ] **Step 6: Checkpoint**
`Task 4b 完成`BrowserDataDir 独立,不复用 distbuild 不会清空状态。
---
## Task 5SessionRegistry含串行化锁
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts
import { SessionRegistry } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js';
describe('SessionRegistry', () => {
let registry: SessionRegistry;
beforeEach(() => { registry = new SessionRegistry(); });
it('register/get/has/list 基础 CRUD', () => {
const fakeCtx = { id: 'A' } as any;
registry.register('s1', fakeCtx);
expect(registry.has('s1')).toBe(true);
expect(registry.get('s1')).toBe(fakeCtx);
expect(registry.list()).toEqual([{ sessionName: 's1', context: fakeCtx, lastUsedAt: expect.any(Date) }]);
registry.unregister('s1');
expect(registry.has('s1')).toBe(false);
});
it('touch 更新 lastUsedAt', async () => {
registry.register('s1', { id: 'A' } as any);
const before = registry.list()[0].lastUsedAt;
await new Promise(r => setTimeout(r, 10));
registry.touch('s1');
const after = registry.list()[0].lastUsedAt;
expect(after.getTime()).toBeGreaterThan(before.getTime());
});
it('withLock: 同 sessionName 串行化', async () => {
const order: string[] = [];
const job = (id: string, ms: number) => async () => {
order.push(`start-${id}`);
await new Promise(r => setTimeout(r, ms));
order.push(`end-${id}`);
};
await Promise.all([
registry.withLock('s1', job('A', 30)),
registry.withLock('s1', job('B', 5)),
registry.withLock('s1', job('C', 5)),
]);
expect(order).toEqual(['start-A', 'end-A', 'start-B', 'end-B', 'start-C', 'end-C']);
});
it('withLock: 不同 sessionName 并发', async () => {
const order: string[] = [];
await Promise.all([
registry.withLock('s1', async () => { order.push('s1-start'); await new Promise(r => setTimeout(r, 30)); order.push('s1-end'); }),
registry.withLock('s2', async () => { order.push('s2-start'); await new Promise(r => setTimeout(r, 10)); order.push('s2-end'); }),
]);
// s2 应该在 s1 完成前先 end
expect(order.indexOf('s2-end')).toBeLessThan(order.indexOf('s1-end'));
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest session-registry
```
预期FAIL
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface SessionEntry {
sessionName: string;
context: any; // patchright BrowserContext
lastUsedAt: Date;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SessionRegistry {
private entries = new Map<string, SessionEntry>();
private locks = new Map<string, Promise<void>>();
register(sessionName: string, context: any): void {
this.entries.set(sessionName, { sessionName, context, lastUsedAt: new Date() });
}
unregister(sessionName: string): void {
this.entries.delete(sessionName);
}
has(sessionName: string): boolean {
return this.entries.has(sessionName);
}
get(sessionName: string): any | undefined {
return this.entries.get(sessionName)?.context;
}
touch(sessionName: string): void {
const e = this.entries.get(sessionName);
if (e) e.lastUsedAt = new Date();
}
list(): SessionEntry[] {
return [...this.entries.values()];
}
/** 同 sessionName 操作严格串行化 */
async withLock<T>(sessionName: string, fn: () => Promise<T>): Promise<T> {
const prev = this.locks.get(sessionName) ?? Promise.resolve();
let release!: () => void;
const next = new Promise<void>(r => (release = r));
this.locks.set(sessionName, prev.then(() => next));
try {
await prev;
return await fn();
} finally {
release();
if (this.locks.get(sessionName) === next) this.locks.delete(sessionName);
}
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest session-registry
```
预期PASS4 tests
- [ ] **Step 5: Checkpoint**
`Task 5 完成`SessionRegistry 含锁机制,并发安全。
---
## Task 6SessionSchedulerLRU 软上限 + 队列)
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts
import { SessionScheduler } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js';
describe('SessionScheduler', () => {
it('未触达上限直接通过', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5 });
sch.recordActivate('s1');
const slot = await sch.acquireSlot('s2', 'normal', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBeUndefined();
});
it('触达上限LRU 回收最久未用的 idle session', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 2, idleTimeoutMs: 1000 });
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 5));
sch.recordActivate('s2');
sch.markIdle('s1');
sch.markIdle('s2');
// s1 比 s2 先 idle所以 s1 是 LRU
const slot = await sch.acquireSlot('s3', 'normal', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBe('s1');
});
it('全部 active 无 idle 可回收fail-fast 503', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1 });
sch.recordActivate('s1');
const slot = await sch.acquireSlot('s2', 'normal', false);
expect(slot.granted).toBe(false);
expect(slot.reason).toBe('NO_IDLE_SESSION_TO_EVICT');
});
it('high 优先级抢占 low 优先级', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1 });
sch.recordActivate('s1', 'low');
const slot = await sch.acquireSlot('s2', 'high', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBe('s1');
});
it('queue=true等待空位最长 30s', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1, queueTimeoutMs: 200 });
sch.recordActivate('s1');
const p = sch.acquireSlot('s2', 'normal', true);
setTimeout(() => sch.markIdle('s1'), 50);
setTimeout(() => sch.recordDeactivate('s1'), 60);
const slot = await p;
expect(slot.granted).toBe(true);
});
it('getStats 返回 active/idle/total', () => {
const sch = new SessionScheduler({ maxActiveSessions: 5 });
sch.recordActivate('s1');
sch.recordActivate('s2');
sch.markIdle('s1');
expect(sch.getStats()).toEqual({
activeCount: 1,
idleCount: 1,
totalCount: 2,
maxActiveSessions: 5,
});
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest session-scheduler
```
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export type Priority = 'low' | 'normal' | 'high';
const PRIORITY_RANK: Record<Priority, number> = { low: 0, normal: 1, high: 2 };
interface SessionState {
sessionName: string;
priority: Priority;
status: 'active' | 'idle';
activatedAt: Date;
lastUsedAt: Date;
}
export interface SchedulerOptions {
maxActiveSessions?: number;
idleTimeoutMs?: number;
queueTimeoutMs?: number;
}
export interface AcquireResult {
granted: boolean;
reason?: string;
evictedSessionName?: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SessionScheduler {
private sessions = new Map<string, SessionState>();
private waiters: Array<{ resolve: (v: AcquireResult) => void; sessionName: string; priority: Priority; expireAt: number }> = [];
private maxActiveSessions: number;
private queueTimeoutMs: number;
constructor(opts: SchedulerOptions = {}) {
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
}
recordActivate(sessionName: string, priority: Priority = 'normal'): void {
const now = new Date();
this.sessions.set(sessionName, { sessionName, priority, status: 'active', activatedAt: now, lastUsedAt: now });
this.tryProcessWaiters();
}
markIdle(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) s.status = 'idle';
this.tryProcessWaiters();
}
recordDeactivate(sessionName: string): void {
this.sessions.delete(sessionName);
this.tryProcessWaiters();
}
touch(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) {
s.status = 'active';
s.lastUsedAt = new Date();
}
}
async acquireSlot(sessionName: string, priority: Priority, queue: boolean): Promise<AcquireResult> {
const direct = this.tryAcquireImmediate(sessionName, priority);
if (direct.granted || !queue) return direct;
if (direct.reason !== 'NO_IDLE_SESSION_TO_EVICT' && direct.reason !== 'AT_CAPACITY') {
return direct;
}
// 排队
return new Promise<AcquireResult>(resolve => {
const expireAt = Date.now() + this.queueTimeoutMs;
const waiter = { resolve, sessionName, priority, expireAt };
this.waiters.push(waiter);
const timer = setTimeout(() => {
const i = this.waiters.indexOf(waiter);
if (i >= 0) {
this.waiters.splice(i, 1);
resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
}
}, this.queueTimeoutMs);
// resolve 时清除 timer
const origResolve = waiter.resolve;
waiter.resolve = (v) => { clearTimeout(timer); origResolve(v); };
});
}
private tryAcquireImmediate(sessionName: string, priority: Priority): AcquireResult {
if (this.sessions.has(sessionName)) {
// 已存在activate 并返回成功
this.touch(sessionName);
return { granted: true };
}
const activeCount = [...this.sessions.values()].filter(s => s.status === 'active').length;
if (activeCount < this.maxActiveSessions) {
return { granted: true };
}
// 触达上限:尝试 LRU 回收 idle
const idleSessions = [...this.sessions.values()]
.filter(s => s.status === 'idle')
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
if (idleSessions.length > 0) {
const evict = idleSessions[0];
this.sessions.delete(evict.sessionName);
return { granted: true, evictedSessionName: evict.sessionName };
}
// 无 idle尝试优先级抢占
const lowerPriority = [...this.sessions.values()]
.filter(s => PRIORITY_RANK[s.priority] < PRIORITY_RANK[priority])
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
if (lowerPriority.length > 0) {
const evict = lowerPriority[0];
this.sessions.delete(evict.sessionName);
return { granted: true, evictedSessionName: evict.sessionName };
}
return { granted: false, reason: 'NO_IDLE_SESSION_TO_EVICT' };
}
private tryProcessWaiters(): void {
while (this.waiters.length > 0) {
const w = this.waiters[0];
if (Date.now() > w.expireAt) {
this.waiters.shift();
w.resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
continue;
}
const r = this.tryAcquireImmediate(w.sessionName, w.priority);
if (r.granted) {
this.waiters.shift();
w.resolve(r);
} else {
break;
}
}
}
getStats(): { activeCount: number; idleCount: number; totalCount: number; maxActiveSessions: number } {
const all = [...this.sessions.values()];
return {
activeCount: all.filter(s => s.status === 'active').length,
idleCount: all.filter(s => s.status === 'idle').length,
totalCount: all.length,
maxActiveSessions: this.maxActiveSessions,
};
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest session-scheduler
```
预期PASS6 tests
- [ ] **Step 5: Checkpoint**
`Task 6 完成`:调度器 LRU+优先级+队列+stats 完整。
---
## Task 7BrowserDaemonService.open/close 基础生命周期
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts`
- [ ] **Step 1: 写测试mock patchright + dependencies**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts
import { BrowserDaemonService } from '../../../../src/modules/netaclaw/browser-daemon/service/daemon.service.js';
describe('BrowserDaemonService.open/close', () => {
let svc: BrowserDaemonService;
let mockChromium: any;
let mockContext: any;
let mockPage: any;
beforeEach(() => {
mockPage = { goto: jest.fn(), close: jest.fn(), url: () => 'about:blank' };
mockContext = {
pages: () => [mockPage],
newPage: jest.fn().mockResolvedValue(mockPage),
close: jest.fn().mockResolvedValue(undefined),
storageState: jest.fn().mockResolvedValue({ cookies: [], origins: [] }),
};
mockChromium = { launchPersistentContext: jest.fn().mockResolvedValue(mockContext) };
svc = new BrowserDaemonService();
(svc as any).registry = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js').SessionRegistry)();
(svc as any).scheduler = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js').SessionScheduler)({ maxActiveSessions: 5 });
(svc as any).fingerprint = { fromSeed: (s: number) => [`--fingerprint=${s}`] };
(svc as any).chromium = mockChromium;
(svc as any).resolveChromiumPath = () => '/fake/chrome.exe';
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
});
it('open: 启动 chromium + 注册 session', async () => {
const r = await svc.open({ sessionName: 's1', url: 'https://example.com', fingerprintSeed: 42 });
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
executablePath: '/fake/chrome.exe',
args: expect.arrayContaining(['--fingerprint=42']),
})
);
expect(r.sessionName).toBe('s1');
expect((svc as any).registry.has('s1')).toBe(true);
});
it('open: sessionName 已存在 → 409', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
await expect(svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }))
.rejects.toThrow(/already exists|conflict/i);
});
it('close: 关闭 context + 注销 session', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
await svc.close('s1');
expect(mockContext.close).toHaveBeenCalled();
expect((svc as any).registry.has('s1')).toBe(false);
});
it('list: 返回所有 session 信息', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
const list = svc.list();
expect(list).toEqual([expect.objectContaining({ sessionName: 's1' })]);
});
it('proxy 配置传给 patchright', async () => {
await svc.open({
sessionName: 's1',
url: 'https://e.com',
fingerprintSeed: 1,
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
});
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
})
);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest daemon.service
```
- [ ] **Step 3: 实现 daemon.service.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts
import { Provide, Scope, ScopeEnum, Inject, Logger, ILogger } from '@midwayjs/core';
import * as path from 'node:path';
import * as fs from 'node:fs';
import { chromium } from 'patchright';
import { SessionRegistry } from '../runtime/session-registry.js';
import { SessionScheduler, Priority } from '../runtime/session-scheduler.js';
import { FingerprintService } from './fingerprint.service.js';
import { resolveChromiumPath, buildLaunchArgs } from '../runtime/chromium-launcher.js';
import { getProfileDir } from '../runtime/browser-data-dir.js';
export type HumanizeMode = 'full' | 'fast' | 'off';
export interface OpenOpts {
sessionName: string;
url: string;
fingerprintSeed?: number;
proxy?: { server: string; username?: string; password?: string };
profileDir?: string;
headed?: boolean;
priority?: Priority;
queue?: boolean;
}
export interface SessionInfo {
sessionName: string;
url: string;
pageCount: number;
status: 'active' | 'idle';
lastUsedAt: Date;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class BrowserDaemonService {
@Inject() registry: SessionRegistry;
@Inject() scheduler: SessionScheduler;
@Inject() fingerprint: FingerprintService;
@Logger() logger: ILogger;
// 注入点便于测试覆盖
protected chromium: any = chromium;
protected resolveChromiumPath = resolveChromiumPath;
/**
* 注v2 修复 P1-8 后 service 不再维护 modeMap保持 stateless
* humanize-mode 完全由调用方每次传:
* - CLI 端:`--mode` 命令级 > `NETA_BROWSER_HUMANIZE_MODE` 环境变量 > 默认 'full'
* - HTTP/Service 调用方:每次方法调用通过参数传入
*/
async open(opts: OpenOpts): Promise<{ sessionName: string; url: string; pageCount: number }> {
return this.registry.withLock(opts.sessionName, async () => {
if (this.registry.has(opts.sessionName)) {
const err: any = new Error(`Session '${opts.sessionName}' already exists`);
err.statusCode = 409;
throw err;
}
const slot = await this.scheduler.acquireSlot(
opts.sessionName,
opts.priority ?? 'normal',
opts.queue ?? false,
);
if (!slot.granted) {
const err: any = new Error(`No slot available: ${slot.reason}`);
err.statusCode = 503;
err.retryAfter = 30;
throw err;
}
if (slot.evictedSessionName) {
await this.closeInternal(slot.evictedSessionName);
}
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
fs.mkdirSync(profilePath, { recursive: true });
const fingerprintArgs = this.fingerprint.fromSeed(opts.fingerprintSeed ?? Math.floor(Math.random() * 100000));
const args = buildLaunchArgs({ fingerprintArgs });
this.logger.info(`[browser-daemon] launching ${opts.sessionName}${opts.url}`);
const ctx = await this.chromium.launchPersistentContext(profilePath, {
executablePath: this.resolveChromiumPath(),
headless: !opts.headed,
args,
proxy: opts.proxy,
});
const page = ctx.pages()[0] ?? (await ctx.newPage());
await page.goto(opts.url);
this.registry.register(opts.sessionName, ctx);
this.scheduler.recordActivate(opts.sessionName, opts.priority ?? 'normal');
return { sessionName: opts.sessionName, url: page.url(), pageCount: ctx.pages().length };
});
}
async close(sessionName: string): Promise<void> {
return this.registry.withLock(sessionName, () => this.closeInternal(sessionName));
}
private async closeInternal(sessionName: string): Promise<void> {
const ctx = this.registry.get(sessionName);
if (!ctx) return;
try {
await ctx.close();
} catch (e: any) {
this.logger.warn(`[browser-daemon] close ${sessionName} failed: ${e.message}`);
}
this.registry.unregister(sessionName);
this.scheduler.recordDeactivate(sessionName);
}
list(): SessionInfo[] {
return this.registry.list().map(e => {
const ctx = e.context;
const page = ctx.pages?.()[0];
return {
sessionName: e.sessionName,
url: page?.url?.() ?? '',
pageCount: ctx.pages?.().length ?? 0,
status: 'active',
lastUsedAt: e.lastUsedAt,
};
});
}
getStats() {
return this.scheduler.getStats();
}
getPageOrFail(sessionName: string) {
const ctx = this.registry.get(sessionName);
if (!ctx) throw new Error(`Session '${sessionName}' not found`);
const page = ctx.pages()[0];
if (!page) throw new Error(`Session '${sessionName}' has no page`);
this.registry.touch(sessionName);
this.scheduler.touch(sessionName);
return { ctx, page };
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest daemon.service
```
预期PASS5 tests
- [ ] **Step 5: Checkpoint**
`Task 7 完成`daemon.service open/close/list 可用含锁、调度、指纹、launch。
---
## Task 8HumanizerService三档拟人化
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts
import { HumanizerService } from '../../../../src/modules/netaclaw/browser-daemon/service/humanizer.service.js';
describe('HumanizerService', () => {
let svc: HumanizerService;
let mockPage: any;
let mockMouse: any;
let mockCursor: any;
beforeEach(() => {
mockMouse = { down: jest.fn(), up: jest.fn(), move: jest.fn(), wheel: jest.fn() };
mockPage = { mouse: mockMouse, waitForTimeout: jest.fn() };
mockCursor = { moveTo: jest.fn().mockResolvedValue(undefined) };
svc = new HumanizerService();
(svc as any).createCursor = () => mockCursor;
(svc as any).rand = (a: number, b: number) => (a + b) / 2; // deterministic
(svc as any).chance = (_: number) => false; // never trigger random branches
});
it('full 模式: ghost-cursor 移动 + 视觉停顿 + mousedown/up', async () => {
await svc.click(mockPage, { x: 100, y: 200 }, 'full');
expect(mockCursor.moveTo).toHaveBeenCalledWith({ x: 100, y: 200 }, expect.any(Object));
expect(mockMouse.down).toHaveBeenCalled();
expect(mockMouse.up).toHaveBeenCalled();
});
it('fast 模式: 不走 ghost-cursor直接 mouse.move + 短延迟', async () => {
await svc.click(mockPage, { x: 100, y: 200 }, 'fast');
expect(mockCursor.moveTo).not.toHaveBeenCalled();
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
expect(mockMouse.down).toHaveBeenCalled();
});
it('off 模式: 立即 click无延迟', async () => {
(svc as any).rand = jest.fn();
await svc.click(mockPage, { x: 100, y: 200 }, 'off');
expect(mockCursor.moveTo).not.toHaveBeenCalled();
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
expect(mockMouse.down).toHaveBeenCalled();
expect((svc as any).rand).not.toHaveBeenCalled();
});
it('type full: 字符间随机间隔5% 概率错字mock 不触发)', async () => {
const keyboard = { type: jest.fn(), press: jest.fn() };
mockPage.keyboard = keyboard;
await svc.type(mockPage, 'hello', 'full');
// 'hello' 5 个字符 → keyboard.type 应该被调 5 次
expect(keyboard.type).toHaveBeenCalledTimes(5);
expect(keyboard.type).toHaveBeenNthCalledWith(1, 'h');
});
it('type off: 直接整体输入', async () => {
const keyboard = { type: jest.fn() };
mockPage.keyboard = keyboard;
await svc.type(mockPage, 'hello', 'off');
expect(keyboard.type).toHaveBeenCalledWith('hello', undefined);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest humanizer.service
```
- [ ] **Step 3: 实现 humanizer.service.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
import { createCursor } from 'ghost-cursor';
export type HumanizeMode = 'full' | 'fast' | 'off';
@Provide()
@Scope(ScopeEnum.Singleton)
export class HumanizerService {
// 注入点便于测试
protected createCursor = createCursor;
protected rand(min: number, max: number): number {
return min + Math.floor(Math.random() * (max - min));
}
protected chance(p: number): boolean {
return Math.random() < p;
}
async click(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
if (mode === 'full') {
// ghost-cursor 是 puppeteer 类型patchright Page 与之不兼容,需要 as any cast
const cursor = this.createCursor(page as any);
await cursor.moveTo(point, { randomizeMoveDelay: true, moveDelay: this.rand(50, 150) });
await this.sleep(this.rand(100, 300));
await page.mouse.down();
await this.sleep(this.rand(50, 150));
await page.mouse.up();
if (this.chance(0.2)) {
await page.mouse.wheel(0, this.rand(2, 10));
}
} else if (mode === 'fast') {
await page.mouse.move(point.x, point.y);
await this.sleep(this.rand(50, 200));
await page.mouse.down();
await this.sleep(this.rand(20, 50));
await page.mouse.up();
} else {
await page.mouse.move(point.x, point.y);
await page.mouse.down();
await page.mouse.up();
}
}
async type(page: any, text: string, mode: HumanizeMode): Promise<void> {
if (mode === 'off') {
await page.keyboard.type(text);
return;
}
const minDelay = mode === 'full' ? 80 : 30;
const maxDelay = mode === 'full' ? 250 : 80;
for (let i = 0; i < text.length; i++) {
const ch = text[i];
// 5% 错字(仅 full 模式)
if (mode === 'full' && this.chance(0.05)) {
const wrong = String.fromCharCode(ch.charCodeAt(0) + 1);
await page.keyboard.type(wrong);
await this.sleep(this.rand(minDelay, maxDelay));
await page.keyboard.press('Backspace');
await this.sleep(this.rand(minDelay, maxDelay));
}
await page.keyboard.type(ch);
await this.sleep(this.rand(minDelay, maxDelay));
}
}
async hover(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
if (mode === 'full') {
const cursor = this.createCursor(page as any);
await cursor.moveTo(point);
await this.sleep(this.rand(200, 800));
} else {
await page.mouse.move(point.x, point.y);
if (mode === 'fast') await this.sleep(this.rand(50, 200));
}
}
async scroll(page: any, deltaY: number, mode: HumanizeMode): Promise<void> {
if (mode === 'off') {
await page.mouse.wheel(0, deltaY);
return;
}
const steps = mode === 'full' ? Math.ceil(Math.abs(deltaY) / 50) : Math.ceil(Math.abs(deltaY) / 200);
const stepSize = deltaY / steps;
const minD = mode === 'full' ? 50 : 10;
const maxD = mode === 'full' ? 150 : 30;
for (let i = 0; i < steps; i++) {
await page.mouse.wheel(0, stepSize);
await this.sleep(this.rand(minD, maxD));
}
}
private sleep(ms: number): Promise<void> {
return new Promise(r => setTimeout(r, ms));
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest humanizer.service
```
预期PASS5 tests
- [ ] **Step 5: Checkpoint**
`Task 8 完成`humanizer 三档行为可用且可测。
---
## Task 9BrowserDaemonService.click/fill/type/snapshot/cookie/state业务方法
**Files:**
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts`
- [ ] **Step 1: 写 SnapshotRefService 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts
import { SnapshotRefService } from '../../../../src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.js';
describe('SnapshotRefService', () => {
let svc: SnapshotRefService;
let mockPage: any;
beforeEach(() => {
svc = new SnapshotRefService();
mockPage = {
evaluate: jest.fn().mockResolvedValue([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'input', text: '' },
]),
};
});
it('snapshot 调 page.evaluate 注入 data-ai-ref返回 refs', async () => {
const refs = await svc.snapshot('s1', mockPage);
expect(refs).toEqual([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'input', text: '' },
]);
expect(mockPage.evaluate).toHaveBeenCalledTimes(1);
});
it('refToSelector 返回 [data-ai-ref="..."]', () => {
expect(svc.refToSelector('e15')).toBe('[data-ai-ref="e15"]');
});
});
```
- [ ] **Step 2: 实现 SnapshotRefService**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface RefEntry {
ref: string;
tag: string;
text: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SnapshotRefService {
/** 给页面所有可交互元素注入 data-ai-ref返回 ref 列表 */
async snapshot(_sessionName: string, page: any): Promise<RefEntry[]> {
return page.evaluate(() => {
const interactive = Array.from(document.querySelectorAll(
'a, button, input, select, textarea, [role="button"], [role="link"], [contenteditable="true"]'
));
const out: { ref: string; tag: string; text: string }[] = [];
interactive.forEach((el, i) => {
const ref = `e${i + 1}`;
el.setAttribute('data-ai-ref', ref);
const tag = el.tagName.toLowerCase();
let text = (el as HTMLElement).innerText || (el as HTMLInputElement).value || '';
text = text.trim().slice(0, 80);
out.push({ ref, tag, text });
});
return out;
});
}
refToSelector(ref: string): string {
return `[data-ai-ref="${ref}"]`;
}
}
```
- [ ] **Step 3: 跑 snapshot-ref 测试**
```bash
pnpm jest snapshot-ref
```
预期PASS
- [ ] **Step 4: 在 BrowserDaemonService 中加业务方法**
`daemon.service.ts` 末尾加入(在 `setHumanizeMode` 之后):
```typescript
// ===== 业务方法(依赖 humanizer + snapshot-ref=====
@Inject() humanizer: HumanizerService;
@Inject() snapshotRef: SnapshotRefService;
async click(sessionName: string, ref: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const locator = page.locator(this.snapshotRef.refToSelector(ref));
const box = await locator.boundingBox();
if (!box) throw new Error(`Ref '${ref}' not visible`);
const m = mode ?? this.getHumanizeMode(sessionName);
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
});
}
async fill(sessionName: string, ref: string, value: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const locator = page.locator(this.snapshotRef.refToSelector(ref));
const box = await locator.boundingBox();
if (!box) throw new Error(`Ref '${ref}' not visible`);
const m = mode ?? this.getHumanizeMode(sessionName);
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
await locator.fill('');
await this.humanizer.type(page, value, m);
});
}
async type(sessionName: string, text: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
await this.humanizer.type(page, text, mode ?? this.getHumanizeMode(sessionName));
});
}
async snapshot(sessionName: string) {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const refs = await this.snapshotRef.snapshot(sessionName, page);
return { url: page.url(), refs };
});
}
async getCookies(sessionName: string, domain?: string): Promise<any[]> {
return this.registry.withLock(sessionName, async () => {
const { ctx } = this.getPageOrFail(sessionName);
const cookies = await ctx.cookies();
return domain ? cookies.filter((c: any) => c.domain.includes(domain)) : cookies;
});
}
async saveState(sessionName: string, filePath: string): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { ctx } = this.getPageOrFail(sessionName);
await ctx.storageState({ path: filePath });
});
}
async loadState(sessionName: string, filePath: string): Promise<void> {
// loadState 需在 launchPersistentContext 时通过 storageState 选项注入
// 但 patchright launchPersistentContext 不直接接受 storageState
// → 实际方案addCookies + 手动 localStorage基于 file 内容)
return this.registry.withLock(sessionName, async () => {
const { ctx, page } = this.getPageOrFail(sessionName);
const fs = await import('node:fs/promises');
const data = JSON.parse(await fs.readFile(filePath, 'utf8'));
if (data.cookies) await ctx.addCookies(data.cookies);
if (data.origins) {
for (const o of data.origins) {
await page.goto(o.origin);
for (const item of o.localStorage ?? []) {
await page.evaluate(([k, v]: [string, string]) => localStorage.setItem(k, v), [item.name, item.value]);
}
}
}
});
}
async goto(sessionName: string, url: string): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
await page.goto(url);
});
}
```
也要在文件顶部加 import
```typescript
import { HumanizerService } from './humanizer.service.js';
import { SnapshotRefService } from './snapshot-ref.service.js';
```
- [ ] **Step 5: 写新方法的测试(追加到 daemon.service.test.ts**
```typescript
// 追加到 daemon.service.test.ts
describe('BrowserDaemonService 业务方法', () => {
let svc: BrowserDaemonService;
let mockCtx: any;
let mockPage: any;
let mockHumanizer: any;
let mockSnapshotRef: any;
beforeEach(() => {
mockPage = {
url: () => 'https://example.com',
mouse: { down: jest.fn(), up: jest.fn(), move: jest.fn() },
keyboard: { type: jest.fn() },
locator: jest.fn().mockReturnValue({
boundingBox: jest.fn().mockResolvedValue({ x: 10, y: 20, width: 100, height: 40 }),
fill: jest.fn(),
}),
goto: jest.fn(),
};
mockCtx = {
pages: () => [mockPage],
cookies: jest.fn().mockResolvedValue([{ name: 'a', value: '1', domain: '.x.com' }]),
storageState: jest.fn(),
addCookies: jest.fn(),
};
mockHumanizer = { click: jest.fn(), type: jest.fn() };
mockSnapshotRef = {
refToSelector: (r: string) => `[data-ai-ref="${r}"]`,
snapshot: jest.fn().mockResolvedValue([{ ref: 'e1', tag: 'button', text: 'OK' }]),
};
svc = new BrowserDaemonService();
const { SessionRegistry } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js');
const { SessionScheduler } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js');
(svc as any).registry = new SessionRegistry();
(svc as any).scheduler = new SessionScheduler({ maxActiveSessions: 5 });
(svc as any).humanizer = mockHumanizer;
(svc as any).snapshotRef = mockSnapshotRef;
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
(svc as any).registry.register('s1', mockCtx);
(svc as any).scheduler.recordActivate('s1');
});
it('click: 调 humanizer.click(中心点, mode)', async () => {
await svc.click('s1', 'e1');
expect(mockHumanizer.click).toHaveBeenCalledWith(mockPage, { x: 60, y: 40 }, 'full');
});
it('snapshot: 调 snapshotRef + 返回 url + refs', async () => {
const r = await svc.snapshot('s1');
expect(r.url).toBe('https://example.com');
expect(r.refs).toEqual([{ ref: 'e1', tag: 'button', text: 'OK' }]);
});
it('getCookies: domain 过滤', async () => {
const r = await svc.getCookies('s1', 'x.com');
expect(r).toEqual([{ name: 'a', value: '1', domain: '.x.com' }]);
});
it('saveState: 调 storageState({path})', async () => {
await svc.saveState('s1', '/tmp/s.json');
expect(mockCtx.storageState).toHaveBeenCalledWith({ path: '/tmp/s.json' });
});
});
```
- [ ] **Step 6: 跑所有 daemon 测试**
```bash
pnpm jest daemon.service
```
预期PASS前 5 tests + 4 新 tests = 9
- [ ] **Step 7: Checkpoint**
`Task 9 完成`BrowserDaemonService 业务方法齐备。
---
## Task 9bCleanup + Lifecycle + Idle-Timeout**新增 P1 修复**
**问题**spec §5.4 要求 @Init 扫 known sessions、@Destroy 优雅关闭 + 5s 超时强制 kill§6.6 要求 60min idle 自动回收。原 plan 未实现。
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts`
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts`
- [ ] **Step 1: 实现 cleanup.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts
/**
* 优雅关闭工具:等待 ctx.close() 最多 5s超时强制 kill
*/
export async function gracefullyClose(ctx: any, timeoutMs = 5000): Promise<void> {
await Promise.race([
ctx.close().catch(() => {}),
new Promise(resolve => setTimeout(resolve, timeoutMs)),
]);
// 强制 kill 残留 chrome 进程
try {
const browser = ctx.browser?.();
if (browser?.process) {
const proc = browser.process();
if (proc && !proc.killed) proc.kill('SIGKILL');
}
} catch {}
}
```
- [ ] **Step 2: SessionScheduler 增加 idle 计时器**
修改 `session-scheduler.ts`,加入:
```typescript
// 在 SessionScheduler 类内
private idleTimers = new Map<string, NodeJS.Timeout>();
private idleTimeoutMs: number;
private onIdleTimeout?: (sessionName: string) => Promise<void>;
constructor(opts: SchedulerOptions = {}) {
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
this.idleTimeoutMs = opts.idleTimeoutMs ?? 60 * 60 * 1000; // 60 min
}
setIdleTimeoutHandler(fn: (sessionName: string) => Promise<void>): void {
this.onIdleTimeout = fn;
}
touch(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) {
s.status = 'active';
s.lastUsedAt = new Date();
}
this.resetIdleTimer(sessionName);
}
private resetIdleTimer(sessionName: string): void {
this.clearIdleTimer(sessionName);
if (this.idleTimeoutMs <= 0 || !this.onIdleTimeout) return;
const timer = setTimeout(() => {
this.idleTimers.delete(sessionName);
this.onIdleTimeout?.(sessionName).catch(() => {});
}, this.idleTimeoutMs);
this.idleTimers.set(sessionName, timer);
}
private clearIdleTimer(sessionName: string): void {
const t = this.idleTimers.get(sessionName);
if (t) {
clearTimeout(t);
this.idleTimers.delete(sessionName);
}
}
// 在 recordActivate 末尾调 resetIdleTimer
// 在 recordDeactivate 调 clearIdleTimer
```
- [ ] **Step 3: daemon.service @Init / @Destroy 钩子**
`daemon.service.ts` 加:
```typescript
import { Init, Destroy } from '@midwayjs/core';
import { gracefullyClose } from '../runtime/cleanup.js';
import { getStateDir } from '../runtime/browser-data-dir.js';
import * as path from 'node:path';
import * as fs from 'node:fs';
// 在 BrowserDaemonService 类内:
@Init()
async init(): Promise<void> {
// 注册 idle 回收回调
this.scheduler.setIdleTimeoutHandler(async (sessionName) => {
this.logger.info(`[browser-daemon] idle timeout: ${sessionName} → save+close`);
await this.registry.withLock(sessionName, async () => {
const ctx = this.registry.get(sessionName);
if (!ctx) return;
const stateFile = getStateDir(sessionName);
try {
await ctx.storageState({ path: stateFile });
} catch (e: any) {
this.logger.warn(`[browser-daemon] saveState on idle failed: ${e.message}`);
}
await this.closeInternal(sessionName);
});
});
// 扫描 known sessions不主动 launch
const stateRoot = path.dirname(getStateDir('placeholder'));
if (fs.existsSync(stateRoot)) {
const files = fs.readdirSync(stateRoot).filter(f => f.endsWith('.json'));
this.logger.info(`[browser-daemon] init: ${files.length} known sessions on disk`);
}
}
@Destroy()
async destroy(): Promise<void> {
this.logger.info('[browser-daemon] destroying, closing all sessions...');
const all = this.registry.list();
await Promise.all(all.map(async (e) => {
try {
const stateFile = getStateDir(e.sessionName);
await e.context.storageState({ path: stateFile });
} catch (err: any) {
this.logger.warn(`[browser-daemon] saveState ${e.sessionName} failed: ${err.message}`);
}
await gracefullyClose(e.context, 5000);
}));
}
```
- [ ] **Step 4: 写 cleanup 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts
import { gracefullyClose } from '../../../../src/modules/netaclaw/browser-daemon/runtime/cleanup.js';
describe('cleanup.gracefullyClose', () => {
it('正常关闭close 完成', async () => {
const close = jest.fn().mockResolvedValue(undefined);
await gracefullyClose({ close }, 1000);
expect(close).toHaveBeenCalled();
});
it('close 超时5s 后强制 kill', async () => {
const kill = jest.fn();
const ctx = {
close: () => new Promise(() => {}), // 永远不 resolve
browser: () => ({ process: () => ({ kill, killed: false }) }),
};
const start = Date.now();
await gracefullyClose(ctx, 100);
expect(Date.now() - start).toBeGreaterThanOrEqual(100);
expect(kill).toHaveBeenCalledWith('SIGKILL');
});
it('close 抛错不阻断', async () => {
await expect(
gracefullyClose({ close: () => Promise.reject(new Error('boom')) }, 100)
).resolves.toBeUndefined();
});
});
```
- [ ] **Step 5: 跑测试**
```bash
pnpm jest cleanup.test
```
预期PASS3 tests
- [ ] **Step 6: SessionScheduler idle timer 测试追加**
```typescript
// 追加到 session-scheduler.test.ts
describe('SessionScheduler idle timer', () => {
it('idle 超时后调 onIdleTimeout', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 50 });
const handler = jest.fn().mockResolvedValue(undefined);
sch.setIdleTimeoutHandler(handler);
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 100));
expect(handler).toHaveBeenCalledWith('s1');
});
it('touch 重置计时器', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 80 });
const handler = jest.fn().mockResolvedValue(undefined);
sch.setIdleTimeoutHandler(handler);
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 50));
sch.touch('s1');
await new Promise(r => setTimeout(r, 50));
// 累积 100ms 但 touch 重置后只过了 50ms未触发
expect(handler).not.toHaveBeenCalled();
await new Promise(r => setTimeout(r, 50));
expect(handler).toHaveBeenCalledWith('s1');
});
});
```
- [ ] **Step 7: Checkpoint**
`Task 9b 完成`lifecycle 钩子 + idle 自动回收 + 优雅关闭。验收 9 + 5b 可通过。
---
## Task 10control-auth.middlewareloopback + secret
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts
import { BrowserControlAuthMiddleware } from '../../../../src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.js';
describe('BrowserControlAuthMiddleware', () => {
const mw = new BrowserControlAuthMiddleware();
const fn = mw.resolve();
beforeEach(() => {
process.env.NETA_TRAY_SECRET = 'test-secret';
process.env.NODE_ENV = 'test';
});
it('loopback + 正确 secret → 通过', async () => {
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(next).toHaveBeenCalled();
});
it('非 loopback → 403', async () => {
const ctx: any = { ip: '8.8.8.8', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(403);
expect(next).not.toHaveBeenCalled();
});
it('loopback + 错误 secret → 401', async () => {
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'wrong' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(401);
});
it('loopback 无 secret → 401', async () => {
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(401);
});
it('dev 模式 + secret 未配置 → 旁路通过loopback 仍校验)', async () => {
process.env.NETA_TRAY_SECRET = '';
process.env.NODE_ENV = 'development';
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(next).toHaveBeenCalled();
});
it('dev 模式 + secret 未配置 + 非 loopback → 仍 403', async () => {
process.env.NETA_TRAY_SECRET = '';
process.env.NODE_ENV = 'development';
const ctx: any = { ip: '8.8.8.8', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(403);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest control-auth.middleware
```
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts
import { IMiddleware, Middleware } from '@midwayjs/core';
import { NextFunction, Context } from '@midwayjs/koa';
import { isLoopbackAddress, validateRuntimeSecret } from '../../../../comm/runtime-secret.js';
@Middleware()
export class BrowserControlAuthMiddleware implements IMiddleware<Context, NextFunction> {
resolve() {
return async (ctx: Context, next: NextFunction) => {
const ip = ctx.ip || (ctx as any).request?.ip || (ctx as any).req?.socket?.remoteAddress;
if (!isLoopbackAddress(ip)) {
ctx.status = 403;
ctx.body = { code: 1003, error: 'Forbidden: only loopback access allowed' };
return;
}
const expected = process.env.NETA_TRAY_SECRET ?? '';
const actual = ctx.headers['x-neta-control-secret'] as string | undefined;
const isDev = process.env.NODE_ENV !== 'production';
// dev 旁路secret 未配置时仅信任 loopback
if (isDev && !expected) {
await next();
return;
}
if (!validateRuntimeSecret(expected, actual)) {
ctx.status = 401;
ctx.body = { code: 1001, error: 'Unauthorized: invalid x-neta-control-secret' };
return;
}
await next();
};
}
match(ctx: Context): boolean {
return ctx.path?.startsWith('/admin/browser-daemon/');
}
static getName(): string {
return 'browserControlAuth';
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest control-auth.middleware
```
预期PASS6 tests
- [ ] **Step 5: 在 netaclaw/config.ts 注册中间件(重要:模块挂载明确)**
修改 `packages/backend/src/modules/netaclaw/config.ts`,把空的 `middlewares: []` 改为:
```typescript
import { ModuleConfig } from '@cool-midway/core';
import { BrowserControlAuthMiddleware } from './browser-daemon/middleware/control-auth.middleware.js';
export default () => {
return {
name: 'NetaClaw',
description: 'NetaClaw 电商浏览器自动化 Agent 引擎',
middlewares: [BrowserControlAuthMiddleware],
globalMiddlewares: [],
order: 0,
} as ModuleConfig;
};
```
> 关键browser-daemon **不是独立模块**,是 netaclaw 的子目录。中间件挂载到 netaclaw 模块,靠 `match()` 限制只在 `/admin/browser-daemon/*` 路径生效。其他 netaclaw 路由(`/admin/netaclaw/*`)不受影响。
注意保留 NetaClawConfig / defaultNetaClawConfig 等导出(在 `// ---` 分隔符之后的部分)。
- [ ] **Step 6: Checkpoint**
`Task 10 完成`auth middleware 校验 loopback + secret含 dev 旁路middleware 已挂载到 netaclaw 模块。
---
## Task 11HTTP Controllers5 个 controller
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts`
> 这 5 个 controller 都是 thin wrapper解析 body → 调 service → 返回结果。**严格按 Neta 现有 controller 风格**
> - `@Provide()` + `@Controller('/admin/browser-daemon')` 类装饰器声明路径前缀
> - 方法用相对路径 `@Post('/open')` `@Get('/list')`
> - 直接返回 `{ code: 1000, data: ... }`,不继承 `BaseController`,不用 `this.ok()`,不用 `@CoolController` 不用 `@CoolTag`
> - 参考 `packages/backend/src/modules/netaclaw/controller/admin/agent_channel.ts` 实际风格
- [ ] **Step 1: 实现 session.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts
import { Provide, Inject, Controller, Post, Get, Body } from '@midwayjs/core';
import { Context } from '@midwayjs/koa';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonSessionController {
@Inject() daemonService: BrowserDaemonService;
@Inject() ctx: Context;
@Post('/open')
async open(@Body() dto: any) {
try {
const r = await this.daemonService.open(dto);
return { code: 1000, data: r };
} catch (e: any) {
this.ctx.status = e.statusCode ?? 400;
return { code: 1002, error: e.message, retryAfter: e.retryAfter };
}
}
@Post('/close')
async close(@Body('sessionName') sessionName: string) {
await this.daemonService.close(sessionName);
return { code: 1000, data: { ok: true } };
}
@Get('/list')
async list() {
return { code: 1000, data: this.daemonService.list() };
}
@Get('/stats')
async stats() {
return { code: 1000, data: this.daemonService.getStats() };
}
}
```
- [ ] **Step 2: 实现 interaction.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonInteractionController {
@Inject() daemonService: BrowserDaemonService;
@Post('/click')
async click(@Body() dto: { sessionName: string; ref: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.click(dto.sessionName, dto.ref, dto.mode ?? 'full');
return { code: 1000, data: { clicked: dto.ref } };
}
@Post('/fill')
async fill(@Body() dto: { sessionName: string; ref: string; value: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.fill(dto.sessionName, dto.ref, dto.value, dto.mode ?? 'full');
return { code: 1000, data: { filled: dto.ref } };
}
@Post('/type')
async type(@Body() dto: { sessionName: string; text: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.type(dto.sessionName, dto.text, dto.mode ?? 'full');
return { code: 1000, data: { typed: dto.text.length } };
}
}
```
- [ ] **Step 3: 实现 navigation.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonNavigationController {
@Inject() daemonService: BrowserDaemonService;
@Post('/goto')
async goto(@Body() dto: { sessionName: string; url: string }) {
await this.daemonService.goto(dto.sessionName, dto.url);
return { code: 1000, data: { navigated: dto.url } };
}
}
```
- [ ] **Step 4: 实现 state.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonStateController {
@Inject() daemonService: BrowserDaemonService;
@Post('/cookie-list')
async cookieList(@Body() dto: { sessionName: string; domain?: string }) {
return { code: 1000, data: await this.daemonService.getCookies(dto.sessionName, dto.domain) };
}
@Post('/state-save')
async stateSave(@Body() dto: { sessionName: string; filePath: string }) {
await this.daemonService.saveState(dto.sessionName, dto.filePath);
return { code: 1000, data: { saved: dto.filePath } };
}
@Post('/state-load')
async stateLoad(@Body() dto: { sessionName: string; filePath: string }) {
await this.daemonService.loadState(dto.sessionName, dto.filePath);
return { code: 1000, data: { loaded: dto.filePath } };
}
}
```
- [ ] **Step 5: 实现 inspect.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonInspectController {
@Inject() daemonService: BrowserDaemonService;
@Post('/snapshot')
async snapshot(@Body() dto: { sessionName: string }) {
return { code: 1000, data: await this.daemonService.snapshot(dto.sessionName) };
}
}
```
- [ ] **Step 6: 启动 backend 验证路由**
dev 模式启动dev 旁路时无需 secret
```bash
cd packages/backend
NODE_ENV=development pnpm dev
```
另一终端:
```bash
# dev 旁路middleware 已实现):仅 loopback 即可secret 未设也能调
curl -X GET http://localhost:8003/admin/browser-daemon/list
```
预期200 + `{"code":1000,"data":[]}`
- [ ] **Step 7: Checkpoint**
`Task 11 完成`5 个 controller 上线HTTP API 可达。
---
## Task 12Contract TestHTTP ↔ service 等价性)
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts`
- [ ] **Step 1: 写 contract test**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts
import { createApp, close } from '@midwayjs/mock';
import { Framework } from '@midwayjs/koa';
describe('BrowserDaemon HTTP ↔ Service 契约', () => {
let app: any;
beforeAll(async () => {
process.env.NETA_TRAY_SECRET = 'ct-secret';
app = await createApp(undefined, undefined, Framework);
});
afterAll(async () => { if (app) await close(app); });
/** Mock daemonService验证 controller 调它的方法名/参数与 HTTP 1:1 */
const cases = [
{ http: { method: 'POST', path: '/admin/browser-daemon/close', body: { sessionName: 's1' } }, service: 'close', args: ['s1'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/click', body: { sessionName: 's1', ref: 'e1', mode: 'full' } }, service: 'click', args: ['s1', 'e1', 'full'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/fill', body: { sessionName: 's1', ref: 'e1', value: 'abc' } }, service: 'fill', args: ['s1', 'e1', 'abc', undefined] },
{ http: { method: 'POST', path: '/admin/browser-daemon/cookie-list', body: { sessionName: 's1', domain: 'x.com' } }, service: 'getCookies', args: ['s1', 'x.com'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/snapshot', body: { sessionName: 's1' } }, service: 'snapshot', args: ['s1'] },
];
for (const c of cases) {
it(`HTTP ${c.http.method} ${c.http.path} → service.${c.service}(${c.args.join(',')})`, async () => {
const fakeService = {
[c.service]: jest.fn().mockResolvedValue(undefined),
};
// 替换 daemonService 实例
const container = (app as any).getApplicationContext();
container.registerObject('browserDaemonService', fakeService);
// 调 HTTP
const supertest = require('supertest');
const res = await supertest(app.getServer())[c.http.method.toLowerCase()](c.http.path)
.set('x-neta-control-secret', 'ct-secret')
.send(c.http.body);
expect(res.status).toBe(200);
expect(fakeService[c.service]).toHaveBeenCalledWith(...c.args);
});
}
});
```
> 注:实施时依据 Midway test 实际 API 调整 `registerObject` / `supertest` 接入;目的是验证 HTTP 与 service 一一对应。
- [ ] **Step 2: 跑 contract test**
```bash
pnpm jest contract.test
```
预期PASS5 cases
- [ ] **Step 3: Checkpoint**
`Task 12 完成`HTTP 与 service 等价性有 contract test 兜底。
---
## Task 13-18netabrowser-cli 包实现(合并描述)
> 这 6 个 task 是 CLI 包内部各文件,模式相似(接受参数 → 调 HTTP → 输出)。每个文件按下面同一套结构实现,不再展开每 step。
### Task 13bin/main.tscommander 解析 + 命令分发)
**Files:**
- Create: `packages/netabrowser-cli/src/bin/main.ts`
- [ ] **Step 1: 实现**
```typescript
#!/usr/bin/env node
// packages/netabrowser-cli/src/bin/main.ts
import { Command } from 'commander';
import { registerSessionCommands } from '../commands/session.js';
import { registerInteractionCommands } from '../commands/interaction.js';
import { registerNavigationCommands } from '../commands/navigation.js';
import { registerStateCommands } from '../commands/state.js';
import { registerInspectCommands } from '../commands/inspect.js';
const program = new Command();
program
.name('netabrowser-cli')
.description('Neta 反风控+拟人化浏览器 CLI')
.version('0.1.0');
registerSessionCommands(program);
registerInteractionCommands(program);
registerNavigationCommands(program);
registerStateCommands(program);
registerInspectCommands(program);
program.parseAsync(process.argv).catch(e => {
console.error('Error:', e.message);
process.exit(1);
});
```
- [ ] **Step 2: Checkpoint**
### Task 14client/runtime-info + http-client
**Files:**
- Create: `packages/netabrowser-cli/src/client/runtime-info.ts`
- Create: `packages/netabrowser-cli/src/client/http-client.ts`
- [ ] **Step 1: runtime-info.ts**
```typescript
// packages/netabrowser-cli/src/client/runtime-info.ts
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
export interface RuntimeInfoLite {
port: number;
controlSecret: string;
controlBaseUrl: string;
}
/**
* v2 修复 P1-6CLI 启动时 cwd 不一定 = backend cwd。
* runtime-info.json 真实写入 packages/backend/dist/runtime-info.json。
* 优先级:环境变量 > pkg 同目录 > monorepo backend dist > user home。
*/
export function readRuntimeInfo(): RuntimeInfoLite {
if (process.env.NETA_RUNTIME_INFO) {
return JSON.parse(fs.readFileSync(process.env.NETA_RUNTIME_INFO, 'utf8'));
}
const candidates = [
// pkg 模式netabrowser-cli.exe 同目录 data/runtime-info.json
path.join(path.dirname(process.execPath), 'data', 'runtime-info.json'),
// dev 模式:从 cli 编译产物上溯找 monorepo再进 backend dist
// __dirname = <root>/packages/netabrowser-cli/dist/client
// 上溯 4 层到 monorepo root再进 packages/backend/dist
path.resolve(__dirname, '../../../../../packages/backend/dist/runtime-info.json'),
// 兜底:用户主目录
path.join(os.homedir(), '.neta', 'runtime-info.json'),
];
for (const p of candidates) {
if (fs.existsSync(p)) return JSON.parse(fs.readFileSync(p, 'utf8'));
}
throw new Error(
`runtime-info.json not found. Set NETA_RUNTIME_INFO env var or run backend first.\nTried:\n${candidates.map(c => ' - ' + c).join('\n')}`
);
}
```
- [ ] **Step 2: http-client.ts**
```typescript
// packages/netabrowser-cli/src/client/http-client.ts
import axios, { AxiosInstance } from 'axios';
import { readRuntimeInfo } from './runtime-info.js';
let client: AxiosInstance | null = null;
export function getHttpClient(): AxiosInstance {
if (client) return client;
const info = readRuntimeInfo();
client = axios.create({
baseURL: info.controlBaseUrl ?? `http://127.0.0.1:${info.port}`,
headers: { 'x-neta-control-secret': info.controlSecret },
timeout: 60_000,
});
return client;
}
export async function callDaemon<T = any>(path: string, body?: any): Promise<T> {
const c = getHttpClient();
const res = await c.post(path, body ?? {});
if (res.data?.code !== 1000) {
throw new Error(`Daemon error: ${res.data?.message ?? 'unknown'}`);
}
return res.data.data as T;
}
```
- [ ] **Step 3: Checkpoint**
### Task 15commands/session.tsopen/close/list
```typescript
// packages/netabrowser-cli/src/commands/session.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerSessionCommands(program: Command) {
program
.command('open <url>')
.description('启动会话并打开 URL')
.requiredOption('--session <name>', 'session 名称')
.option('--proxy <url>', '代理 URLhttp://user:pass@host:port')
.option('--fingerprint-seed <n>', '指纹 seed', parseInt)
.option('--profile-dir <name>', 'profile 目录名')
.option('--headed', '有头模式')
.option('--humanize-mode <mode>', 'full|fast|off', 'full')
.option('--priority <p>', 'low|normal|high', 'normal')
.option('--queue', '排队等位')
.option('--raw', '输出 raw JSON')
.action(async (url, opts) => {
const dto: any = {
sessionName: opts.session,
url,
humanizeMode: opts.humanizeMode,
priority: opts.priority,
queue: !!opts.queue,
headed: !!opts.headed,
};
if (opts.fingerprintSeed != null) dto.fingerprintSeed = opts.fingerprintSeed;
if (opts.profileDir) dto.profileDir = opts.profileDir;
if (opts.proxy) dto.proxy = parseProxyUrl(opts.proxy);
const r = await callDaemon('/admin/browser-daemon/open', dto);
console.log(formatOutput(r, opts.raw));
});
program
.command('close')
.requiredOption('--session <name>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/close', { sessionName: opts.session });
console.log(formatOutput({ ok: true }, opts.raw));
});
program
.command('list')
.description('列出所有会话')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/list');
console.log(formatOutput(r, opts.raw));
});
}
function parseProxyUrl(url: string) {
const u = new URL(url);
return {
server: `${u.protocol}//${u.host}`,
username: decodeURIComponent(u.username) || undefined,
password: decodeURIComponent(u.password) || undefined,
};
}
```
- [ ] **Checkpoint**
### Task 16commands/interaction.ts
```typescript
// packages/netabrowser-cli/src/commands/interaction.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerInteractionCommands(program: Command) {
program
.command('click <ref>')
.requiredOption('--session <name>')
.option('--mode <mode>', 'full|fast|off')
.option('--raw')
.action(async (ref, opts) => {
// v2 修复 P1-8CLI 端 mode 优先级 = 命令级 --mode > NETA_BROWSER_HUMANIZE_MODE 环境变量 > 默认 full
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/click', { sessionName: opts.session, ref, mode });
console.log(formatOutput({ clicked: ref }, opts.raw));
});
program
.command('fill <ref> <value>')
.requiredOption('--session <name>')
.option('--mode <mode>')
.option('--raw')
.action(async (ref, value, opts) => {
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/fill', { sessionName: opts.session, ref, value, mode });
console.log(formatOutput({ filled: ref }, opts.raw));
});
program
.command('type <text>')
.requiredOption('--session <name>')
.option('--mode <mode>')
.option('--raw')
.action(async (text, opts) => {
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/type', { sessionName: opts.session, text, mode });
console.log(formatOutput({ typed: text.length }, opts.raw));
});
}
```
- [ ] **Checkpoint**
### Task 17commands/navigation + state + inspect
```typescript
// packages/netabrowser-cli/src/commands/navigation.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerNavigationCommands(program: Command) {
program
.command('goto <url>')
.requiredOption('--session <name>')
.option('--raw')
.action(async (url, opts) => {
await callDaemon('/admin/browser-daemon/goto', { sessionName: opts.session, url });
console.log(formatOutput({ navigated: url }, opts.raw));
});
}
```
```typescript
// packages/netabrowser-cli/src/commands/state.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerStateCommands(program: Command) {
program
.command('cookie-list')
.requiredOption('--session <name>')
.option('--domain <d>')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/cookie-list', { sessionName: opts.session, domain: opts.domain });
console.log(formatOutput(r, opts.raw));
});
program
.command('state-save')
.requiredOption('--session <name>')
.requiredOption('--output <path>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/state-save', { sessionName: opts.session, filePath: opts.output });
console.log(formatOutput({ saved: opts.output }, opts.raw));
});
program
.command('state-load')
.requiredOption('--session <name>')
.requiredOption('--input <path>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/state-load', { sessionName: opts.session, filePath: opts.input });
console.log(formatOutput({ loaded: opts.input }, opts.raw));
});
}
```
```typescript
// packages/netabrowser-cli/src/commands/inspect.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerInspectCommands(program: Command) {
program
.command('snapshot')
.requiredOption('--session <name>')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/snapshot', { sessionName: opts.session });
console.log(formatOutput(r, opts.raw));
});
}
```
- [ ] **Checkpoint**
### Task 18output/formatter.ts
```typescript
// packages/netabrowser-cli/src/output/formatter.ts
export function formatOutput(data: any, raw?: boolean): string {
if (raw) return JSON.stringify(data);
if (Array.isArray(data) && data.length > 0 && data[0]?.ref) {
// snapshot 友好输出
return data.map((r: any) => `[${r.ref}] ${r.tag}: ${r.text}`).join('\n');
}
return JSON.stringify(data, null, 2);
}
```
写最小测试:
```typescript
// packages/netabrowser-cli/tests/output-formatter.test.ts
import { formatOutput } from '../src/output/formatter.js';
describe('formatter', () => {
it('--raw → 单行 JSON', () => {
expect(formatOutput({ a: 1 }, true)).toBe('{"a":1}');
});
it('snapshot refs → 友好输出', () => {
const r = formatOutput([{ ref: 'e1', tag: 'button', text: 'OK' }]);
expect(r).toContain('[e1] button: OK');
});
});
```
- [ ] **StepCheckpoint**
---
## Task 19SKILL.mdNetaClaw Agent skill 元数据)
**Files:**
- Create: `packages/backend/skills/netabrowser-cli/SKILL.md`
```markdown
---
name: netabrowser-cli
description: Neta 反风控+拟人化浏览器自动化 CLI。养号、电商自动化、AI 探索复杂网页时使用。比 playwright-cli 更难被反风控识别fingerprint-chromium 内核 + patchright 反自动化 + ghost-cursor 拟人化轨迹。100 账号矩阵养号场景首选。
allowed-tools: Bash(netabrowser-cli:*) Bash(npx:*)
---
# Browser Automation with netabrowser-cli
## When to use vs playwright-cli vs patchwright-cli
- **netabrowser-cli**:国内反风控严的站(小红书/抖音/淘宝/拼多多/微博)+ 多账号矩阵 + 养号场景
- **patchwright-cli**:国外 Cloudflare/DataDome 等反爬保护的站
- **playwright-cli**:测试、无反风控的站
If targeting Chinese social/e-commerce sites or running multi-account automation, **prefer netabrowser-cli**.
## Quick start
```bash
# 启动会话(每会话独立指纹+独立 IP
netabrowser-cli open https://www.xiaohongshu.com \
--session=acc1 \
--proxy=http://user:pass@host:port \
--fingerprint-seed=12345 \
--headed
# 拿当前页面可交互元素
netabrowser-cli snapshot --session=acc1
# 输出:[e1] button: 登录 / [e2] input: 手机号 / ...
# 拟人化点击 + 输入(默认 humanize-mode=full
netabrowser-cli click e1 --session=acc1
netabrowser-cli fill e2 13800001234 --session=acc1
# 抓 cookie
netabrowser-cli cookie-list --session=acc1 --domain=xiaohongshu.com
# 保存登录态
netabrowser-cli state-save --session=acc1 --output=/path/to/state.json
# 关闭
netabrowser-cli close --session=acc1
```
## Commands
### Session 管理
- `open <url>` 启动会话
- `close` 关闭
- `list` 列出所有 session
### 交互(默认拟人化 full
- `click <ref>` 点击 ref 元素(贝塞尔轨迹+视觉停顿+mousedown/up
- `fill <ref> <value>` focus + 拟人化输入
- `type <text>` 在已 focused 输入框打字
- `--mode=fast` 单命令切批量模式200-500ms 延迟,无轨迹)
- `--mode=off` 测试用,立即执行
### 导航
- `goto <url>` 跳转
### 状态
- `cookie-list [--domain=x]`
- `state-save --output=path` 保存完整登录态cookie+localStorage
- `state-load --input=path` 恢复
### 检查
- `snapshot` 拿可交互元素 ref 列表
## 参数规则
### 指纹
- `--fingerprint-seed=N` 单一 seed 派生所有指纹维度(推荐)
- 不同 seed → 不同 canvas/webgl/UA fingerprint同 seed → 完全可复现
### 代理
- `--proxy=http://user:pass@host:port` 单参数最方便
- 出口 IP 实测 = 代理 IPpatchright `launchPersistentContext` 验证通过)
### 拟人化档位
| `--humanize-mode` | 单命令开销 | 适用 |
|---|---|---|
| full默认 | 2-5s | 养号、敏感操作、AI 探索 |
| fast | 0.3-0.7s | 批量发布、批量评论 |
| off | <100ms | 测试CI |
### 调度
- `--priority=low\|normal\|high` 优先级
- `--queue` 触达上限时排队最长 30s
## 性能特征(重要)
养号场景每号约 30 操作 × 5s 2.5min100 号串行 4 小时批量发布请用 `--humanize-mode=fast`
## 详细文档
- [拟人化档位详解](references/humanization.md)
- [指纹参数](references/fingerprint.md)
- [代理配置](references/proxy.md)
- [典型场景示例](references/examples.md)
```
- [ ] **StepCheckpoint**
## Task 20references/ 子文档
> 创建 4 个 reference 文件作为 SKILL.md 的扩展。每个文件 50-100 行,按 SKILL.md 中链接对应的主题展开。
**Files:**
- Create: `packages/backend/skills/netabrowser-cli/references/humanization.md`
- Create: `packages/backend/skills/netabrowser-cli/references/fingerprint.md`
- Create: `packages/backend/skills/netabrowser-cli/references/proxy.md`
- Create: `packages/backend/skills/netabrowser-cli/references/examples.md`
每个文件简明阐述对应主题,含示例命令。具体内容由实施者从 spec §6.1-6.6 + 实测脚本提炼。
- [ ] **StepCheckpoint**
---
## Task 21Windows 打包集成
**Files:**
- Modify: `packages/backend/scripts/pkg-build.js`
- Modify: `packages/backend/scripts/build-windows-installer.js`
- Modify: `packages/backend/installer/setup.iss`
- [ ] **Step 1: pkg-build.js 增加 netabrowser-cli 编译**
读 `pkg-build.js` 现有逻辑,在 backend.exe 编译之后增加:
```javascript
// 编译 netabrowser-cli
console.log('Building netabrowser-cli...');
const cliRoot = path.resolve(__dirname, '../../netabrowser-cli');
execSync('pnpm build', { cwd: cliRoot, stdio: 'inherit' });
// pkg 打包
const pkgConfig = {
name: 'netabrowser-cli',
bin: path.join(cliRoot, 'dist', 'bin', 'main.js'),
targets: ['node22-win-x64'],
outputPath: path.join(__dirname, '../installer/dist/netabrowser-cli.exe'),
};
await pkg(pkgConfig);
```
- [ ] **Step 2: setup.iss 增加 [Files] 段**
在现有 `[Files]` 后追加
```iss
Source: "dist/netabrowser-cli.exe"; DestDir: "{app}"; Flags: ignoreversion
Source: "..\..\netabrowser-cli\chromium\win64\*"; DestDir: "{app}\chromium\win64"; Flags: ignoreversion recursesubdirs createallsubdirs
Source: "..\..\backend\skills\netabrowser-cli\*"; DestDir: "{app}\skills\netabrowser-cli"; Flags: ignoreversion recursesubdirs createallsubdirs
```
- [ ] **Step 3: build-windows-installer.js 加 chromium 复制步骤**
在调 ISCC 之前确保 `chromium/win64` 存在
- [ ] **Step 4: 实跑一次安装包构建dev 机器)**
```bash
cd packages/backend
pnpm run build:windows-installer
```
预期生成的 setup.exe 大小 ~250-400MB装到测试 Windows能找到 `C:\Program Files\Neta\netabrowser-cli.exe` `chromium/win64/chrome.exe`
- [ ] **Step 5: Checkpoint**
---
## Task 22联调与冒烟手工验收
spec §9 验收清单逐条跑
- [ ] **Step 1: 跑所有单测**
```bash
cd packages/backend
pnpm jest test/modules/netaclaw/browser-daemon/
```
预期 PASS
- [ ] **Step 2: 启动 backend dev 模式**
```bash
NODE_ENV=development pnpm dev
```
预期runtime-info.json 写入 `packages/backend/dist/runtime-info.json` controlSecretdev 模式下 secret 可能为空字符串middleware 已开 dev 旁路
> v2 注dev 模式下 NETA_TRAY_SECRET 通常未设。middleware 已实现 dev 旁路:`NODE_ENV !== 'production' && expected === ''` → 仅校验 loopback、跳过 secret 校验。
- [ ] **Step 3: cli list 应返回空**
```bash
node packages/netabrowser-cli/dist/bin/main.js list --raw
```
预期`[]`
- [ ] **Step 4: 真代理出口 IP 验证**
```bash
node packages/netabrowser-cli/dist/bin/main.js open https://httpbin.org/ip \
--session=test \
--proxy="http://e50b26:7ecdccfd@210.51.27.112:10000" \
--headed
```
然后
```bash
node packages/netabrowser-cli/dist/bin/main.js snapshot --session=test
```
url + 内容含 `"origin": "210.51.27.112"`
- [ ] **Step 5: creepjs 反检测 trust score**
打开 https://abrahamjuliot.github.io/creepjs/肉眼记 trust score预期 70%。
- [ ] **Step 6: 拟人化档位延迟测量**
```bash
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=full
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=fast
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=off
```
预期full ~2-5s, fast ~0.3-0.7s, off <0.2s
- [ ] **Step 7: HTTP 直接调,验证 auth**
**dev 旁路验证**
```bash
# dev 模式NODE_ENV=developmentNETA_TRAY_SECRET 未设)
curl http://localhost:8003/admin/browser-daemon/list
# 200dev 旁路允许 loopback
# 非 loopback 仍 403
curl --interface 8.8.8.8 http://localhost:8003/admin/browser-daemon/list
# 403
```
**prod 风格验证**
```bash
# 杀掉 backend重新启动 prod 模式
NODE_ENV=production NETA_TRAY_SECRET=test-prod-secret pnpm start
# 无 secret → 401
curl http://localhost:8003/admin/browser-daemon/list
# 401
# 错误 secret → 401
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: wrong"
# 401
# 正确 secret → 200
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: test-prod-secret"
# 200
```
- [ ] **Step 8: 软上限验证**
`maxActiveSessions=2`启动 3 session 看第 3 个返回 503
- [ ] **Step 9: backend 重启 ref 失效**
ctrl+C backend重启 `pnpm dev` `click e1 --session=test`
预期报错session 不存在或 ref 失效需要先 `open` `snapshot` `click`
- [ ] **Step 10: 安装包冒烟**
setup.exe 装到全新 Windows启动托盘 后端启动 cli step 4-5
- [ ] **Step 11: Checkpoint**
`Task 22 完成`所有验收标准通过
---
## Task 23文档更新 + 路线图
**Files:**
- Modify: `docs/superpowers/specs/2026-05-04-netabrowser-cli-s1-design.md`变更日志
- Modify: `docs/superpowers/specs/2026-05-03-geo-master-roadmap.md`如存在则更新
- [ ] **Step 1: 更新 spec §13 变更日志**
加一行"2026-XX-XX 实施完成所有验收标准通过准备进入 S2"。
- [ ] **Step 2: 更新 geo 路线图**
如有 geo master-roadmap标注 netabrowser-cli S1 完成 解锁 geo BrowserAutomationService 迁移
- [ ] **Step 3: 提示 user 走完整 git commit**
```bash
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
git status
# 用户审核后统一 commit如其要求
```
- [ ] **Step 4: Checkpoint**
`Task 23 完成`文档同步等待 user 触发统一 commit
---
## 实施期间约束
- 不写 SQL 文件如果有数据库需求 mcp mysql 工具直接 INSERT
- 每个 task **不单独 git commit**用户要求全部完成 + 联调通过后统一提交
- 不修改现有 playwright-cli / patchwright-cli skill
- 不在 controller 中加业务逻辑只能是 thin wrapper,§5.3.1 强制
- HTTP API 必须强制 loopback + secret不能例外
- 不实现平台特定补丁小红书/抖音/淘宝等)→ S2
---
## 实施完成后
- 调用 `superpowers:verification-before-completion` 逐条核对验收标准
- 调用 `superpowers:requesting-code-review` 触发代码审查
- 用户测试通过后统一 git commit
- 更新 spec §13 变更日志
- 进入下一个子项目geo BrowserAutomationService 迁移到 BrowserDaemonServiceS2