GPU_GUARD_MONOREPO/docs/superpowers/plans/2026-05-04-netabrowser-cli-s1-plan.md

3174 lines
108 KiB
Markdown
Raw Normal View History

2026-05-20 21:39:12 +08:00
# netabrowser-cli S1 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
>
> **上位文档**[`../specs/2026-05-04-netabrowser-cli-s1-design.md`](../specs/2026-05-04-netabrowser-cli-s1-design.md)
> **Git 策略**:用户要求"全部完成 + 联调通过后统一提交"。每 task 末尾标注 `Checkpoint`(仅作进度标记,不做 git commit
**Goal:** 在 Neta monorepo 内交付 netabrowser-cli S1 基础设施BrowserDaemonService嵌入 backend+ netabrowser-cli 二进制 + skill 元数据,提供反风控+拟人化的浏览器自动化能力,统一服务于 NetaClaw Agent 探索期CLI和后端业务固化期@Inject service
**Architecture:** Service-First 架构:核心是 backend 内的 `BrowserDaemonService`Midway @Singleton)持有 `Map<sessionName, BrowserContext>`CLI 是 thin client 通过 HTTP loopback + secret 调 backend底层用 `patchright` (npm) 启动 vendored `neta-chromium` (fingerprint-chromium);拟人化用 `ghost-cursor` + 自加套,分 full/fast/off 三档session-name 串行化锁LRU 软上限调度。
**Tech Stack:** Midway.js 3.20 / TypeScript 5.9 / `patchright` (npm) / `ghost-cursor` (npm) / fingerprint-chromium 144.0.7559.132 (vendored 二进制) / commander (CLI 解析) / jest+ts-jest / Inno Setup / yao-pkg
---
## ⚠️ v2 修复总览(架构 review 后必读)
本 plan 经过架构师交叉验证发现 8 项 P0/P1 问题,已就地修复。实施时优先理解这些修复点:
| # | 问题 | 修复落点 |
|---|---|---|
| **P0-1** | Controller 路径与 Cool Admin 风格冲突 | Task 11 全部 controller 改用 `@Provide()` + `@Controller('/admin/browser-daemon/<sub>')` + 方法相对路径,参考 `netaclaw/controller/admin/agent_channel.ts` 风格 |
| **P0-2** | dev 模式 dataDir = `<cwd>/dist`build 清空状态 | 新增 Task 4b`resolveBrowserDataDir()` 独立解析dev 模式 → `<monorepo>/.netabrowser-data/`prod → `<dataDir>/.netabrowser/` |
| **P0-3** | browser-daemon 模块挂载错位 | Task 10 Step 5 明确:把 `BrowserControlAuthMiddleware` 加到 `netaclaw/config.ts``middlewares` 数组,依靠 `match()` 仅作用于 `/admin/browser-daemon/*` 路径 |
| **P0-4** | ghost-cursor 类型与 patchright 不兼容 | Task 8 Step 3 humanizer 内部 `createCursor(page as any)` cast |
| **P1-5** | dev 模式 NETA_TRAY_SECRET 未设 → 全 401 | Task 10 middleware 加 dev 旁路:`process.env.NODE_ENV !== 'production' && expected === ''` 时跳过 secret 校验 |
| **P1-6** | CLI runtime-info 路径错位cwd 不对) | Task 14 候选改为 `<monorepo-root>/packages/backend/dist/runtime-info.json` + 强制 `NETA_RUNTIME_INFO` 环境变量优先 |
| **P1-7** | @Init/@Destroy lifecycle + 60min idle 自动回收缺失 | 新增 Task 9b`runtime/cleanup.ts` + daemon.service `@Init`/`@Destroy` + scheduler idle timer |
| **P1-8** | CLI 与会话级 mode 协同 bug | service 移除 modeMapservice-statelessCLI 端用环境变量 `NETA_BROWSER_HUMANIZE_MODE` 作"会话级默认",命令级 `--mode` 覆盖 |
**额外路径修正**Task 4 `chromium-launcher` `__dirname` 上溯 **7 层**(不是 6 层)—— 实际路径 `dist/modules/netaclaw/browser-daemon/runtime/` 到 monorepo root 是 7 个 `..`
**版本锁定**Task 1 安装命令改为 `pnpm add patchright@1.59.4 ghost-cursor@1.4.2`(已验证兼容版本)。
---
## 文件结构
### 后端模块(新增 14 个文件)
| 路径 | 职责 |
|---|---|
| `packages/backend/src/modules/netaclaw/browser-daemon/config.ts` | 模块配置 + `maxActiveSessions` 等 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` | ★ BrowserDaemonService 核心,统一锁 + 调度入口 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts` | 拟人化包装ghost-cursor + 三档) |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts` | neta-chromium 指纹参数生成 |
| `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts` | AI ref 协议spike #1 选定方案) |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts` | 路径解析 + patchright launch 包装 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts` | Map<name, context> + 串行化锁 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` | LRU 软上限调度 |
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts` | @Destroy 时优雅关闭 |
| `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts` | loopback + secret auth |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts` | /open /close /list /stats |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts` | /click /fill /type /scroll /hover /press |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts` | /goto /back /forward /reload |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts` | /cookie-list /cookie-set /state-save /state-load |
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts` | /snapshot /screenshot /eval /run-code |
### netabrowser-cli 包(新增 13 个文件)
| 路径 | 职责 |
|---|---|
| `packages/netabrowser-cli/package.json` | bin 声明 + patchright/commander 等依赖 |
| `packages/netabrowser-cli/tsconfig.json` | TS 配置 |
| `packages/netabrowser-cli/src/bin/main.ts` | CLI 入口commander 解析) |
| `packages/netabrowser-cli/src/client/runtime-info.ts` | 读 backend runtime-info.json |
| `packages/netabrowser-cli/src/client/http-client.ts` | HTTP 请求 + secret header |
| `packages/netabrowser-cli/src/output/formatter.ts` | --raw vs 默认 + 错误格式化 |
| `packages/netabrowser-cli/src/commands/session.ts` | open/close/list共用一个文件 |
| `packages/netabrowser-cli/src/commands/interaction.ts` | click/fill/type/scroll/hover/press |
| `packages/netabrowser-cli/src/commands/navigation.ts` | goto/back/forward/reload |
| `packages/netabrowser-cli/src/commands/state.ts` | cookie / state |
| `packages/netabrowser-cli/src/commands/inspect.ts` | snapshot/screenshot/eval/run-code |
### 测试文件(新增 11 个)
| 路径 | 测试对象 |
|---|---|
| `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts` | 路径解析 + launch args 拼装 |
| `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts` | seed → args 派生 |
| `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts` | 三档行为 + 注入 random source 验证延迟范围 |
| `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts` | 串行化锁竞态 |
| `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts` | LRU + 软上限 + 队列 |
| `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts` | ref 协议契约 |
| `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts` | 编排open/click/cookie/close mock 全链路 |
| `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts` | loopback + secret 校验 |
| `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts` | HTTP ↔ service 等价性 contract |
| `packages/netabrowser-cli/tests/cli-parse.test.ts` | commander 参数解析 |
| `packages/netabrowser-cli/tests/output-formatter.test.ts` | 输出格式 |
### Skill 元数据(新增 5 个文件)
| 路径 | 职责 |
|---|---|
| `packages/backend/skills/netabrowser-cli/SKILL.md` | 给 NetaClaw Agent 看的命令清单 |
| `packages/backend/skills/netabrowser-cli/references/humanization.md` | 三档详解 |
| `packages/backend/skills/netabrowser-cli/references/fingerprint.md` | 指纹参数 |
| `packages/backend/skills/netabrowser-cli/references/proxy.md` | 代理配置 |
| `packages/backend/skills/netabrowser-cli/references/examples.md` | 典型场景 |
### 集成改动
| 路径 | 改动 |
|---|---|
| `pnpm-workspace.yaml` | 新增 `packages/netabrowser-cli` |
| `packages/backend/package.json` | 新增 `patchright` + `ghost-cursor` 依赖 |
| `packages/backend/installer/setup.iss` | `[Files]` 段加 chromium/win64 + netabrowser-cli.exe |
| `packages/backend/scripts/build-windows-installer.js` | 复制 chromium 二进制步骤 |
| `packages/backend/scripts/pkg-build.js` | 增加 netabrowser-cli pkg 编译 |
---
## 实施依赖图
```
Phase 0: T0(包脚手架) → T1(npm 依赖)
Phase 1: T2(Spike #1 ref 协议) ‖ T3(Spike #2 ghost-cursor 兼容) ← 阻塞门
Phase 2: T4(launcher) → T5(registry+lock) → T6(scheduler) → T7(open/close)
T8(humanizer + click/fill) → T9(state/cookie/snapshot)
Phase 3: T10(auth middleware) → T11(controllers) → T12(contract test)
Phase 4: T13(cli bin) → T14(client) → T15-17(命令实现) → T18(formatter)
Phase 5: T19-20(skill 元数据)
Phase 6: T21(pkg build) → T22(installer) → T23(路径解析)
Phase 7: T24(联调冒烟) → T25(性能验证) → T26(文档)
```
---
## Task 0netabrowser-cli 包脚手架
**Files:**
- Create: `packages/netabrowser-cli/package.json`
- Create: `packages/netabrowser-cli/tsconfig.json`
- Modify: `pnpm-workspace.yaml`
- [ ] **Step 1: 把 netabrowser-cli 加入 pnpm workspace**
`pnpm-workspace.yaml`,确认含有 `packages/*` 通配符(应该已有)。如无 `packages/*` 通配符,加上:
```yaml
packages:
- 'packages/*'
```
无需修改的话跳过。
- [ ] **Step 2: 创建 netabrowser-cli/package.json**
```json
{
"name": "@neta/netabrowser-cli",
"version": "0.1.0",
"description": "Neta 反风控+拟人化浏览器自动化 CLI",
"private": true,
"type": "module",
"bin": {
"netabrowser-cli": "./dist/bin/main.js"
},
"scripts": {
"build": "tsc -p tsconfig.json",
"test": "jest"
},
"dependencies": {
"commander": "^12.0.0",
"axios": "^1.7.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"jest": "^29.7.0",
"ts-jest": "^29.1.5",
"typescript": "^5.9.0"
}
}
```
- [ ] **Step 3: 创建 netabrowser-cli/tsconfig.json**
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"resolveJsonModule": true,
"declaration": false
},
"include": ["src/**/*"]
}
```
- [ ] **Step 4: 验证 pnpm 识别 workspace**
```bash
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
pnpm install
pnpm list -r --depth=-1 | grep netabrowser-cli
```
预期:列表中出现 `@neta/netabrowser-cli`
- [ ] **Step 5: Checkpoint**
`Task 0 完成`:包注册到 workspace可以独立 build/test。
---
## Task 1backend 安装 patchright + ghost-cursor 依赖
**Files:**
- Modify: `packages/backend/package.json`
- [ ] **Step 1: 在 backend 安装 patchrightnpm 包,锁定版本)**
```bash
cd packages/backend
pnpm add patchright@1.59.4
```
> 锁定 1.59.4:经 review 验证此版本兼容 Node 22 + Midway 3.20 + neta-chromium 144.0.7559.132。后续升级前需重测兼容性。
- [ ] **Step 2: 安装 ghost-cursor锁定版本**
```bash
pnpm add ghost-cursor@1.4.2
```
- [ ] **Step 3: 验证 patchright 能 import**
跑一段临时脚本:
```bash
cd packages/backend
node -e "import('patchright').then(p => console.log('patchright loaded:', Object.keys(p).join(',')))"
```
预期输出含 `chromium,firefox,webkit,...`
- [ ] **Step 4: 验证 ghost-cursor 能 import**
```bash
node -e "import('ghost-cursor').then(p => console.log('ghost-cursor loaded:', Object.keys(p).join(',')))"
```
预期含 `createCursor,path,...`
- [ ] **Step 5: Checkpoint**
`Task 1 完成`:依赖装好,可以在 service 代码 import 使用。
---
## Task 2 [SPIKE #1]AI Ref 协议方案验证
> 这是 spec §11 的阻塞门 spike。**必须先做,否则后续 interaction 命令无法实现**。
**目标**:选定一种 AI ref 协议方案并 demo 跑通。
候选方案:
- **A. 自实现**snapshot 时给 DOM 注入 `data-ai-ref="e15"` 属性 + 内存维护 `Map<sessionName, Map<ref, ElementHandle>>`
- **B. 改用 selector**:命令直接接收 selector 字符串(`click 'button:has-text("登录")'`),不要 ref
- **C. vendor playwright-cli ref 代码**:剥离 playwright-cli 内部 ref 实现引入 netabrowser-cli
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts`
- [ ] **Step 1: 写 spike 测试(方案 A先验证最简实现**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts
import { chromium } from 'patchright';
import * as path from 'node:path';
import * as fs from 'node:fs';
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
describe('Spike #1: AI ref 协议(方案 A 自实现)', () => {
jest.setTimeout(60000);
it('A: snapshot 注入 data-ai-ref可基于 ref 定位元素', async () => {
if (!fs.existsSync(CHROME)) {
console.warn('SKIP: neta-chromium not found at', CHROME);
return;
}
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-1');
const ctx = await chromium.launchPersistentContext(tmpProfile, {
executablePath: CHROME,
headless: true,
args: ['--fingerprint=12345'],
});
const page = ctx.pages()[0] || (await ctx.newPage());
await page.goto('data:text/html,<button id=login>Login</button><a href=#>Link</a>');
// 1. snapshot给所有可交互元素注入 data-ai-ref
const refs = await page.evaluate(() => {
const interactive = Array.from(document.querySelectorAll('button, a, input, select, textarea'));
const out: { ref: string; tag: string; text: string }[] = [];
interactive.forEach((el, i) => {
const ref = `e${i + 1}`;
el.setAttribute('data-ai-ref', ref);
out.push({ ref, tag: el.tagName.toLowerCase(), text: (el.textContent || '').trim() });
});
return out;
});
expect(refs).toEqual([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'a', text: 'Link' },
]);
// 2. 通过 ref click
await page.locator('[data-ai-ref="e1"]').click();
// 不报错 = 通过
await ctx.close();
fs.rmSync(tmpProfile, { recursive: true, force: true });
});
});
```
- [ ] **Step 2: 跑 spike 测试**
```bash
cd packages/backend
pnpm jest test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts -t 'A:'
```
预期PASS含 fingerprint chromium 已在 vendor 里)
- [ ] **Step 3: 评审 spike 结果**
如果方案 A PASS写入决策文件
```bash
cat > packages/backend/src/modules/netaclaw/browser-daemon/SPIKE_DECISION.md << 'EOF'
# Spike #1 决策AI Ref 协议
**选定方案 A自实现 data-ai-ref 注入**
## 理由
- 实现简单(一段 page.evaluate JS 即可)
- 不引入额外依赖
- 完全可控
## 实现要点
1. snapshot 命令时调用 `injectRefs(page)`,给所有可交互元素加 `data-ai-ref="eN"`
2. service 内存维护 `Map<sessionName, Map<ref, locator-info>>` 用于审计/调试
3. interaction 命令收到 ref 后用 `page.locator('[data-ai-ref="${ref}"]')` 定位
4. 页面跳转/重新 snapshot 后旧 ref 失效DOM 变化)
## 不选 B/C 的理由
- BselectorAI 写 selector 容易错ref 抽象更友好
- Cvendor playwright-cliplaywright-cli 用 socket+state 同步,移植成本高
EOF
```
如果方案 A FAIL极不可能降级 B/C 重写 spike。
- [ ] **Step 4: Checkpoint**
`Task 2 完成`:方案 A 通过,决策记录在 `SPIKE_DECISION.md`。后续 snapshot-ref.service.ts 实现该方案。
---
## Task 3 [SPIKE #2]ghost-cursor on patchright 兼容性验证
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts`
- [ ] **Step 1: 写 spike 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
import { chromium } from 'patchright';
import { createCursor } from 'ghost-cursor';
import * as path from 'node:path';
import * as fs from 'node:fs';
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
describe('Spike #2: ghost-cursor on patchright + neta-chromium', () => {
jest.setTimeout(60000);
it('cursor.click 能在 patchright 启动的 neta-chromium 上工作', async () => {
if (!fs.existsSync(CHROME)) {
console.warn('SKIP: neta-chromium not found');
return;
}
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-2');
const ctx = await chromium.launchPersistentContext(tmpProfile, {
executablePath: CHROME,
headless: true,
args: ['--fingerprint=99999'],
});
const page = ctx.pages()[0] || (await ctx.newPage());
await page.goto('data:text/html,<button id=b style="position:absolute;top:200px;left:300px">Click me</button><script>document.getElementById("b").addEventListener("click",()=>document.title="CLICKED")</script>');
// 创建 ghost cursor
const cursor = createCursor(page as any);
// 拿目标坐标
const box = await page.locator('#b').boundingBox();
expect(box).not.toBeNull();
// 用 ghost-cursor 移动并点击(贝塞尔轨迹)
await cursor.moveTo({ x: box!.x + box!.width / 2, y: box!.y + box!.height / 2 });
await page.mouse.down();
await page.waitForTimeout(80);
await page.mouse.up();
// 等待 click 生效
await page.waitForFunction(() => document.title === 'CLICKED', null, { timeout: 5000 });
expect(await page.title()).toBe('CLICKED');
// 验证 navigator.webdriver 仍为 falsepatchright 反检测有效)
const wd = await page.evaluate(() => navigator.webdriver);
expect(wd).toBe(false);
await ctx.close();
fs.rmSync(tmpProfile, { recursive: true, force: true });
});
});
```
- [ ] **Step 2: 跑 spike**
```bash
pnpm jest test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
```
预期PASS含 navigator.webdriver === false
- [ ] **Step 3: 写决策**
如果通过append `SPIKE_DECISION.md`
```markdown
# Spike #2 决策ghost-cursor 兼容
✅ ghost-cursor 在 patchright + neta-chromium 上工作正常。
- cursor.moveTo 走贝塞尔轨迹patchright 没拦截 mouse 事件
- click 后 navigator.webdriver 仍 falsepatchright 反检测维持)
humanizer.service.ts 直接 `import { createCursor } from 'ghost-cursor'` 即可。
```
如果失败:选用纯自写贝塞尔(替代方案,备选脚本另准备)。
- [ ] **Step 4: Checkpoint**
`Task 3 完成`ghost-cursor 兼容性确认,可投入 humanizer 实现。
---
## Task 4chromium-launcher路径解析 + launch args 拼装)
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts`
- [ ] **Step 1: 写 fingerprint 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts
import { FingerprintService } from '../../../../src/modules/netaclaw/browser-daemon/service/fingerprint.service.js';
describe('FingerprintService', () => {
const svc = new FingerprintService();
it('fromSeed: 仅 seed 生成完整 args', () => {
const args = svc.fromSeed(12345);
expect(args).toEqual(expect.arrayContaining([
'--fingerprint=12345',
'--fingerprint-platform=Windows',
'--fingerprint-platform-version=10.0.0',
'--fingerprint-brand=Chrome',
'--fingerprint-hardware-concurrency=8',
'--fingerprint-language=zh-CN',
]));
});
it('merge: 指定字段覆盖默认', () => {
const args = svc.merge({ seed: 1, language: 'en-US', hardwareConcurrency: 16 });
expect(args).toContain('--fingerprint=1');
expect(args).toContain('--fingerprint-language=en-US');
expect(args).toContain('--fingerprint-hardware-concurrency=16');
});
it('seed 不同生成不同 fingerprint arg', () => {
expect(svc.fromSeed(1)).toContain('--fingerprint=1');
expect(svc.fromSeed(2)).toContain('--fingerprint=2');
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
cd packages/backend && pnpm jest fingerprint.service
```
预期FAILservice 不存在)
- [ ] **Step 3: 实现 FingerprintService**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface FingerprintParams {
seed?: number;
platform?: 'Windows' | 'macOS' | 'Linux';
platformVersion?: string;
brand?: string;
brandVersion?: string;
hardwareConcurrency?: number;
language?: string;
timezone?: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class FingerprintService {
/** 仅 seed → 全套默认 args */
fromSeed(seed: number): string[] {
return this.merge({ seed });
}
/** 自定义参数 + 默认值 → args */
merge(p: FingerprintParams): string[] {
const args: string[] = [];
if (p.seed != null) args.push(`--fingerprint=${p.seed}`);
args.push(`--fingerprint-platform=${p.platform ?? 'Windows'}`);
args.push(`--fingerprint-platform-version=${p.platformVersion ?? '10.0.0'}`);
args.push(`--fingerprint-brand=${p.brand ?? 'Chrome'}`);
if (p.brandVersion) args.push(`--fingerprint-brand-version=${p.brandVersion}`);
args.push(`--fingerprint-hardware-concurrency=${p.hardwareConcurrency ?? 8}`);
args.push(`--fingerprint-language=${p.language ?? 'zh-CN'}`);
if (p.timezone) args.push(`--fingerprint-timezone=${p.timezone}`);
return args;
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest fingerprint.service
```
预期PASS3 tests
- [ ] **Step 5: 写 chromium-launcher 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts
import { resolveChromiumPath, buildLaunchArgs } from '../../../../src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.js';
import * as path from 'node:path';
describe('chromium-launcher', () => {
describe('resolveChromiumPath', () => {
afterEach(() => { delete process.env.NETA_CHROMIUM_PATH; });
it('优先环境变量', () => {
process.env.NETA_CHROMIUM_PATH = '/custom/chrome.exe';
expect(resolveChromiumPath({ isPkg: false })).toBe('/custom/chrome.exe');
});
it('pkg 模式execPath 同目录 chromium/win64/chrome.exe', () => {
const result = resolveChromiumPath({ isPkg: true, execPath: 'C:/Program Files/Neta/backend.exe' });
expect(result).toMatch(/Program Files\/Neta\/chromium\/win64\/chrome\.exe$/);
});
it('dev 模式monorepo packages/netabrowser-cli/chromium/win64/chrome.exe', () => {
const result = resolveChromiumPath({ isPkg: false });
expect(result).toMatch(/netabrowser-cli\/chromium\/win64\/chrome\.exe$/);
});
});
describe('buildLaunchArgs', () => {
it('合并指纹 args + 用户自定义 args', () => {
const args = buildLaunchArgs({
fingerprintArgs: ['--fingerprint=42', '--fingerprint-language=zh-CN'],
extraArgs: ['--disable-popup-blocking'],
});
expect(args).toEqual(expect.arrayContaining([
'--fingerprint=42',
'--fingerprint-language=zh-CN',
'--disable-popup-blocking',
]));
});
});
});
```
- [ ] **Step 6: 实现 chromium-launcher**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts
import * as path from 'node:path';
export interface ResolveOpts {
isPkg?: boolean;
execPath?: string;
}
/** 解析 neta-chromium 可执行文件路径 */
export function resolveChromiumPath(opts: ResolveOpts = {}): string {
if (process.env.NETA_CHROMIUM_PATH) return process.env.NETA_CHROMIUM_PATH;
const isPkg = opts.isPkg ?? !!(process as any).pkg;
if (isPkg) {
const execPath = opts.execPath ?? process.execPath;
return path.join(path.dirname(execPath), 'chromium', 'win64', 'chrome.exe').replace(/\\/g, '/');
}
// dev 模式:从 backend src 出发,定位 monorepo 内 packages/netabrowser-cli/chromium/win64/chrome.exe
// 编译后 __dirname = packages/backend/dist/modules/netaclaw/browser-daemon/runtime
// 路径计算runtime → browser-daemon → netaclaw → modules → dist → backend → packages → root7 层 ..
// 然后 root/packages/netabrowser-cli/chromium/win64/chrome.exe
return path.resolve(__dirname, '../../../../../../../packages/netabrowser-cli/chromium/win64/chrome.exe').replace(/\\/g, '/');
}
export interface LaunchArgsOpts {
fingerprintArgs: string[];
extraArgs?: string[];
}
export function buildLaunchArgs(opts: LaunchArgsOpts): string[] {
return [...opts.fingerprintArgs, ...(opts.extraArgs ?? [])];
}
```
- [ ] **Step 7: 跑 chromium-launcher 测试**
```bash
pnpm jest chromium-launcher
```
预期PASS4 tests
- [ ] **Step 8: Checkpoint**
`Task 4 完成`:路径解析 + 指纹参数生成 + launch args 拼装可用。
---
## Task 4bBrowserDataDir 独立解析(**新增 P0 修复**
**问题**`comm/data-dir.ts:resolveDataDir()` 在 dev 模式 fallback 到 `<cwd>/dist`。把 `browser-profiles/` `states/` 放 dist 下,每次 `pnpm build` 会清空,状态全丢。
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts
import { resolveBrowserDataDir, getProfileDir, getStateDir } from '../../../../src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.js';
import * as path from 'node:path';
describe('BrowserDataDir', () => {
afterEach(() => {
delete process.env.NETA_BROWSER_DATA_DIR;
delete process.env.NETA_DATA_DIR;
});
it('优先 NETA_BROWSER_DATA_DIR 环境变量', () => {
process.env.NETA_BROWSER_DATA_DIR = '/custom/browser';
expect(resolveBrowserDataDir({ isPkg: false })).toBe('/custom/browser');
});
it('pkg 模式dataDir/.netabrowser', () => {
const r = resolveBrowserDataDir({ isPkg: true, execDir: 'C:/Program Files/Neta' });
expect(r.replace(/\\/g, '/')).toBe('C:/Program Files/Neta/data/.netabrowser');
});
it('dev 模式monorepo/.netabrowser-data不复用 dist', () => {
const r = resolveBrowserDataDir({ isPkg: false, monorepoRoot: '/repo' });
expect(r.replace(/\\/g, '/')).toBe('/repo/.netabrowser-data');
});
it('getProfileDir / getStateDir 返回子路径', () => {
process.env.NETA_BROWSER_DATA_DIR = '/x';
expect(getProfileDir('s1').replace(/\\/g, '/')).toBe('/x/profiles/s1');
expect(getStateDir('s1').replace(/\\/g, '/')).toBe('/x/states/s1.json');
});
});
```
- [ ] **Step 2: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts
import * as path from 'node:path';
import { resolveDataDir } from '../../../../comm/data-dir.js';
export interface ResolveBrowserDataDirOpts {
isPkg?: boolean;
execDir?: string;
monorepoRoot?: string;
}
/**
* netabrowser-cli 自己的状态根目录,独立于 backend dist/。
* - prod: <dataDir>/.netabrowser/
* - dev: <monorepo-root>/.netabrowser-data/ (不放 dist 下,防 build 清空)
*/
export function resolveBrowserDataDir(opts: ResolveBrowserDataDirOpts = {}): string {
if (process.env.NETA_BROWSER_DATA_DIR) return path.resolve(process.env.NETA_BROWSER_DATA_DIR);
const isPkg = opts.isPkg ?? !!(process as any).pkg;
if (isPkg) {
const dataDir = resolveDataDir({ isPkg: true, execDir: opts.execDir });
return path.join(dataDir, '.netabrowser');
}
// dev: 从 backend 出发上溯到 monorepo root
const monorepoRoot = opts.monorepoRoot
?? path.resolve(__dirname, '../../../../../../../');
return path.join(monorepoRoot, '.netabrowser-data');
}
export function getProfileDir(sessionName: string): string {
return path.join(resolveBrowserDataDir(), 'profiles', sessionName);
}
export function getStateDir(sessionName: string): string {
return path.join(resolveBrowserDataDir(), 'states', `${sessionName}.json`);
}
```
- [ ] **Step 3: 跑测试看通过**
```bash
pnpm jest browser-data-dir
```
预期PASS4 tests
- [ ] **Step 4: 修改 Task 7 daemon.service 中的 import**
将 daemon.service.ts 的:
```typescript
import { resolveDataDir } from '../../../../comm/data-dir.js';
// ...
const profileRoot = path.join(resolveDataDir(), 'browser-profiles');
const profilePath = path.join(profileRoot, opts.profileDir ?? opts.sessionName);
```
改为:
```typescript
import { getProfileDir } from '../runtime/browser-data-dir.js';
// ...
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
```
同样将 saveState/loadState 用到的状态文件路径改为 `getStateDir(sessionName)`
- [ ] **Step 5: 更新 .gitignore**
`.netabrowser-data/` 加入 monorepo 根 `.gitignore`
```bash
echo ".netabrowser-data/" >> C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo/.gitignore
```
- [ ] **Step 6: Checkpoint**
`Task 4b 完成`BrowserDataDir 独立,不复用 distbuild 不会清空状态。
---
## Task 5SessionRegistry含串行化锁
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts
import { SessionRegistry } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js';
describe('SessionRegistry', () => {
let registry: SessionRegistry;
beforeEach(() => { registry = new SessionRegistry(); });
it('register/get/has/list 基础 CRUD', () => {
const fakeCtx = { id: 'A' } as any;
registry.register('s1', fakeCtx);
expect(registry.has('s1')).toBe(true);
expect(registry.get('s1')).toBe(fakeCtx);
expect(registry.list()).toEqual([{ sessionName: 's1', context: fakeCtx, lastUsedAt: expect.any(Date) }]);
registry.unregister('s1');
expect(registry.has('s1')).toBe(false);
});
it('touch 更新 lastUsedAt', async () => {
registry.register('s1', { id: 'A' } as any);
const before = registry.list()[0].lastUsedAt;
await new Promise(r => setTimeout(r, 10));
registry.touch('s1');
const after = registry.list()[0].lastUsedAt;
expect(after.getTime()).toBeGreaterThan(before.getTime());
});
it('withLock: 同 sessionName 串行化', async () => {
const order: string[] = [];
const job = (id: string, ms: number) => async () => {
order.push(`start-${id}`);
await new Promise(r => setTimeout(r, ms));
order.push(`end-${id}`);
};
await Promise.all([
registry.withLock('s1', job('A', 30)),
registry.withLock('s1', job('B', 5)),
registry.withLock('s1', job('C', 5)),
]);
expect(order).toEqual(['start-A', 'end-A', 'start-B', 'end-B', 'start-C', 'end-C']);
});
it('withLock: 不同 sessionName 并发', async () => {
const order: string[] = [];
await Promise.all([
registry.withLock('s1', async () => { order.push('s1-start'); await new Promise(r => setTimeout(r, 30)); order.push('s1-end'); }),
registry.withLock('s2', async () => { order.push('s2-start'); await new Promise(r => setTimeout(r, 10)); order.push('s2-end'); }),
]);
// s2 应该在 s1 完成前先 end
expect(order.indexOf('s2-end')).toBeLessThan(order.indexOf('s1-end'));
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest session-registry
```
预期FAIL
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface SessionEntry {
sessionName: string;
context: any; // patchright BrowserContext
lastUsedAt: Date;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SessionRegistry {
private entries = new Map<string, SessionEntry>();
private locks = new Map<string, Promise<void>>();
register(sessionName: string, context: any): void {
this.entries.set(sessionName, { sessionName, context, lastUsedAt: new Date() });
}
unregister(sessionName: string): void {
this.entries.delete(sessionName);
}
has(sessionName: string): boolean {
return this.entries.has(sessionName);
}
get(sessionName: string): any | undefined {
return this.entries.get(sessionName)?.context;
}
touch(sessionName: string): void {
const e = this.entries.get(sessionName);
if (e) e.lastUsedAt = new Date();
}
list(): SessionEntry[] {
return [...this.entries.values()];
}
/** 同 sessionName 操作严格串行化 */
async withLock<T>(sessionName: string, fn: () => Promise<T>): Promise<T> {
const prev = this.locks.get(sessionName) ?? Promise.resolve();
let release!: () => void;
const next = new Promise<void>(r => (release = r));
this.locks.set(sessionName, prev.then(() => next));
try {
await prev;
return await fn();
} finally {
release();
if (this.locks.get(sessionName) === next) this.locks.delete(sessionName);
}
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest session-registry
```
预期PASS4 tests
- [ ] **Step 5: Checkpoint**
`Task 5 完成`SessionRegistry 含锁机制,并发安全。
---
## Task 6SessionSchedulerLRU 软上限 + 队列)
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts
import { SessionScheduler } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js';
describe('SessionScheduler', () => {
it('未触达上限直接通过', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5 });
sch.recordActivate('s1');
const slot = await sch.acquireSlot('s2', 'normal', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBeUndefined();
});
it('触达上限LRU 回收最久未用的 idle session', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 2, idleTimeoutMs: 1000 });
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 5));
sch.recordActivate('s2');
sch.markIdle('s1');
sch.markIdle('s2');
// s1 比 s2 先 idle所以 s1 是 LRU
const slot = await sch.acquireSlot('s3', 'normal', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBe('s1');
});
it('全部 active 无 idle 可回收fail-fast 503', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1 });
sch.recordActivate('s1');
const slot = await sch.acquireSlot('s2', 'normal', false);
expect(slot.granted).toBe(false);
expect(slot.reason).toBe('NO_IDLE_SESSION_TO_EVICT');
});
it('high 优先级抢占 low 优先级', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1 });
sch.recordActivate('s1', 'low');
const slot = await sch.acquireSlot('s2', 'high', false);
expect(slot.granted).toBe(true);
expect(slot.evictedSessionName).toBe('s1');
});
it('queue=true等待空位最长 30s', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 1, queueTimeoutMs: 200 });
sch.recordActivate('s1');
const p = sch.acquireSlot('s2', 'normal', true);
setTimeout(() => sch.markIdle('s1'), 50);
setTimeout(() => sch.recordDeactivate('s1'), 60);
const slot = await p;
expect(slot.granted).toBe(true);
});
it('getStats 返回 active/idle/total', () => {
const sch = new SessionScheduler({ maxActiveSessions: 5 });
sch.recordActivate('s1');
sch.recordActivate('s2');
sch.markIdle('s1');
expect(sch.getStats()).toEqual({
activeCount: 1,
idleCount: 1,
totalCount: 2,
maxActiveSessions: 5,
});
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest session-scheduler
```
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export type Priority = 'low' | 'normal' | 'high';
const PRIORITY_RANK: Record<Priority, number> = { low: 0, normal: 1, high: 2 };
interface SessionState {
sessionName: string;
priority: Priority;
status: 'active' | 'idle';
activatedAt: Date;
lastUsedAt: Date;
}
export interface SchedulerOptions {
maxActiveSessions?: number;
idleTimeoutMs?: number;
queueTimeoutMs?: number;
}
export interface AcquireResult {
granted: boolean;
reason?: string;
evictedSessionName?: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SessionScheduler {
private sessions = new Map<string, SessionState>();
private waiters: Array<{ resolve: (v: AcquireResult) => void; sessionName: string; priority: Priority; expireAt: number }> = [];
private maxActiveSessions: number;
private queueTimeoutMs: number;
constructor(opts: SchedulerOptions = {}) {
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
}
recordActivate(sessionName: string, priority: Priority = 'normal'): void {
const now = new Date();
this.sessions.set(sessionName, { sessionName, priority, status: 'active', activatedAt: now, lastUsedAt: now });
this.tryProcessWaiters();
}
markIdle(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) s.status = 'idle';
this.tryProcessWaiters();
}
recordDeactivate(sessionName: string): void {
this.sessions.delete(sessionName);
this.tryProcessWaiters();
}
touch(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) {
s.status = 'active';
s.lastUsedAt = new Date();
}
}
async acquireSlot(sessionName: string, priority: Priority, queue: boolean): Promise<AcquireResult> {
const direct = this.tryAcquireImmediate(sessionName, priority);
if (direct.granted || !queue) return direct;
if (direct.reason !== 'NO_IDLE_SESSION_TO_EVICT' && direct.reason !== 'AT_CAPACITY') {
return direct;
}
// 排队
return new Promise<AcquireResult>(resolve => {
const expireAt = Date.now() + this.queueTimeoutMs;
const waiter = { resolve, sessionName, priority, expireAt };
this.waiters.push(waiter);
const timer = setTimeout(() => {
const i = this.waiters.indexOf(waiter);
if (i >= 0) {
this.waiters.splice(i, 1);
resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
}
}, this.queueTimeoutMs);
// resolve 时清除 timer
const origResolve = waiter.resolve;
waiter.resolve = (v) => { clearTimeout(timer); origResolve(v); };
});
}
private tryAcquireImmediate(sessionName: string, priority: Priority): AcquireResult {
if (this.sessions.has(sessionName)) {
// 已存在activate 并返回成功
this.touch(sessionName);
return { granted: true };
}
const activeCount = [...this.sessions.values()].filter(s => s.status === 'active').length;
if (activeCount < this.maxActiveSessions) {
return { granted: true };
}
// 触达上限:尝试 LRU 回收 idle
const idleSessions = [...this.sessions.values()]
.filter(s => s.status === 'idle')
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
if (idleSessions.length > 0) {
const evict = idleSessions[0];
this.sessions.delete(evict.sessionName);
return { granted: true, evictedSessionName: evict.sessionName };
}
// 无 idle尝试优先级抢占
const lowerPriority = [...this.sessions.values()]
.filter(s => PRIORITY_RANK[s.priority] < PRIORITY_RANK[priority])
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
if (lowerPriority.length > 0) {
const evict = lowerPriority[0];
this.sessions.delete(evict.sessionName);
return { granted: true, evictedSessionName: evict.sessionName };
}
return { granted: false, reason: 'NO_IDLE_SESSION_TO_EVICT' };
}
private tryProcessWaiters(): void {
while (this.waiters.length > 0) {
const w = this.waiters[0];
if (Date.now() > w.expireAt) {
this.waiters.shift();
w.resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
continue;
}
const r = this.tryAcquireImmediate(w.sessionName, w.priority);
if (r.granted) {
this.waiters.shift();
w.resolve(r);
} else {
break;
}
}
}
getStats(): { activeCount: number; idleCount: number; totalCount: number; maxActiveSessions: number } {
const all = [...this.sessions.values()];
return {
activeCount: all.filter(s => s.status === 'active').length,
idleCount: all.filter(s => s.status === 'idle').length,
totalCount: all.length,
maxActiveSessions: this.maxActiveSessions,
};
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest session-scheduler
```
预期PASS6 tests
- [ ] **Step 5: Checkpoint**
`Task 6 完成`:调度器 LRU+优先级+队列+stats 完整。
---
## Task 7BrowserDaemonService.open/close 基础生命周期
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts`
- [ ] **Step 1: 写测试mock patchright + dependencies**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts
import { BrowserDaemonService } from '../../../../src/modules/netaclaw/browser-daemon/service/daemon.service.js';
describe('BrowserDaemonService.open/close', () => {
let svc: BrowserDaemonService;
let mockChromium: any;
let mockContext: any;
let mockPage: any;
beforeEach(() => {
mockPage = { goto: jest.fn(), close: jest.fn(), url: () => 'about:blank' };
mockContext = {
pages: () => [mockPage],
newPage: jest.fn().mockResolvedValue(mockPage),
close: jest.fn().mockResolvedValue(undefined),
storageState: jest.fn().mockResolvedValue({ cookies: [], origins: [] }),
};
mockChromium = { launchPersistentContext: jest.fn().mockResolvedValue(mockContext) };
svc = new BrowserDaemonService();
(svc as any).registry = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js').SessionRegistry)();
(svc as any).scheduler = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js').SessionScheduler)({ maxActiveSessions: 5 });
(svc as any).fingerprint = { fromSeed: (s: number) => [`--fingerprint=${s}`] };
(svc as any).chromium = mockChromium;
(svc as any).resolveChromiumPath = () => '/fake/chrome.exe';
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
});
it('open: 启动 chromium + 注册 session', async () => {
const r = await svc.open({ sessionName: 's1', url: 'https://example.com', fingerprintSeed: 42 });
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
executablePath: '/fake/chrome.exe',
args: expect.arrayContaining(['--fingerprint=42']),
})
);
expect(r.sessionName).toBe('s1');
expect((svc as any).registry.has('s1')).toBe(true);
});
it('open: sessionName 已存在 → 409', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
await expect(svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }))
.rejects.toThrow(/already exists|conflict/i);
});
it('close: 关闭 context + 注销 session', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
await svc.close('s1');
expect(mockContext.close).toHaveBeenCalled();
expect((svc as any).registry.has('s1')).toBe(false);
});
it('list: 返回所有 session 信息', async () => {
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
const list = svc.list();
expect(list).toEqual([expect.objectContaining({ sessionName: 's1' })]);
});
it('proxy 配置传给 patchright', async () => {
await svc.open({
sessionName: 's1',
url: 'https://e.com',
fingerprintSeed: 1,
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
});
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
})
);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest daemon.service
```
- [ ] **Step 3: 实现 daemon.service.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts
import { Provide, Scope, ScopeEnum, Inject, Logger, ILogger } from '@midwayjs/core';
import * as path from 'node:path';
import * as fs from 'node:fs';
import { chromium } from 'patchright';
import { SessionRegistry } from '../runtime/session-registry.js';
import { SessionScheduler, Priority } from '../runtime/session-scheduler.js';
import { FingerprintService } from './fingerprint.service.js';
import { resolveChromiumPath, buildLaunchArgs } from '../runtime/chromium-launcher.js';
import { getProfileDir } from '../runtime/browser-data-dir.js';
export type HumanizeMode = 'full' | 'fast' | 'off';
export interface OpenOpts {
sessionName: string;
url: string;
fingerprintSeed?: number;
proxy?: { server: string; username?: string; password?: string };
profileDir?: string;
headed?: boolean;
priority?: Priority;
queue?: boolean;
}
export interface SessionInfo {
sessionName: string;
url: string;
pageCount: number;
status: 'active' | 'idle';
lastUsedAt: Date;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class BrowserDaemonService {
@Inject() registry: SessionRegistry;
@Inject() scheduler: SessionScheduler;
@Inject() fingerprint: FingerprintService;
@Logger() logger: ILogger;
// 注入点便于测试覆盖
protected chromium: any = chromium;
protected resolveChromiumPath = resolveChromiumPath;
/**
*v2 修复 P1-8 后 service 不再维护 modeMap保持 stateless
* humanize-mode 完全由调用方每次传:
* - CLI 端:`--mode` 命令级 > `NETA_BROWSER_HUMANIZE_MODE` 环境变量 > 默认 'full'
* - HTTP/Service 调用方:每次方法调用通过参数传入
*/
async open(opts: OpenOpts): Promise<{ sessionName: string; url: string; pageCount: number }> {
return this.registry.withLock(opts.sessionName, async () => {
if (this.registry.has(opts.sessionName)) {
const err: any = new Error(`Session '${opts.sessionName}' already exists`);
err.statusCode = 409;
throw err;
}
const slot = await this.scheduler.acquireSlot(
opts.sessionName,
opts.priority ?? 'normal',
opts.queue ?? false,
);
if (!slot.granted) {
const err: any = new Error(`No slot available: ${slot.reason}`);
err.statusCode = 503;
err.retryAfter = 30;
throw err;
}
if (slot.evictedSessionName) {
await this.closeInternal(slot.evictedSessionName);
}
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
fs.mkdirSync(profilePath, { recursive: true });
const fingerprintArgs = this.fingerprint.fromSeed(opts.fingerprintSeed ?? Math.floor(Math.random() * 100000));
const args = buildLaunchArgs({ fingerprintArgs });
this.logger.info(`[browser-daemon] launching ${opts.sessionName} → ${opts.url}`);
const ctx = await this.chromium.launchPersistentContext(profilePath, {
executablePath: this.resolveChromiumPath(),
headless: !opts.headed,
args,
proxy: opts.proxy,
});
const page = ctx.pages()[0] ?? (await ctx.newPage());
await page.goto(opts.url);
this.registry.register(opts.sessionName, ctx);
this.scheduler.recordActivate(opts.sessionName, opts.priority ?? 'normal');
return { sessionName: opts.sessionName, url: page.url(), pageCount: ctx.pages().length };
});
}
async close(sessionName: string): Promise<void> {
return this.registry.withLock(sessionName, () => this.closeInternal(sessionName));
}
private async closeInternal(sessionName: string): Promise<void> {
const ctx = this.registry.get(sessionName);
if (!ctx) return;
try {
await ctx.close();
} catch (e: any) {
this.logger.warn(`[browser-daemon] close ${sessionName} failed: ${e.message}`);
}
this.registry.unregister(sessionName);
this.scheduler.recordDeactivate(sessionName);
}
list(): SessionInfo[] {
return this.registry.list().map(e => {
const ctx = e.context;
const page = ctx.pages?.()[0];
return {
sessionName: e.sessionName,
url: page?.url?.() ?? '',
pageCount: ctx.pages?.().length ?? 0,
status: 'active',
lastUsedAt: e.lastUsedAt,
};
});
}
getStats() {
return this.scheduler.getStats();
}
getPageOrFail(sessionName: string) {
const ctx = this.registry.get(sessionName);
if (!ctx) throw new Error(`Session '${sessionName}' not found`);
const page = ctx.pages()[0];
if (!page) throw new Error(`Session '${sessionName}' has no page`);
this.registry.touch(sessionName);
this.scheduler.touch(sessionName);
return { ctx, page };
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest daemon.service
```
预期PASS5 tests
- [ ] **Step 5: Checkpoint**
`Task 7 完成`daemon.service open/close/list 可用含锁、调度、指纹、launch。
---
## Task 8HumanizerService三档拟人化
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts
import { HumanizerService } from '../../../../src/modules/netaclaw/browser-daemon/service/humanizer.service.js';
describe('HumanizerService', () => {
let svc: HumanizerService;
let mockPage: any;
let mockMouse: any;
let mockCursor: any;
beforeEach(() => {
mockMouse = { down: jest.fn(), up: jest.fn(), move: jest.fn(), wheel: jest.fn() };
mockPage = { mouse: mockMouse, waitForTimeout: jest.fn() };
mockCursor = { moveTo: jest.fn().mockResolvedValue(undefined) };
svc = new HumanizerService();
(svc as any).createCursor = () => mockCursor;
(svc as any).rand = (a: number, b: number) => (a + b) / 2; // deterministic
(svc as any).chance = (_: number) => false; // never trigger random branches
});
it('full 模式: ghost-cursor 移动 + 视觉停顿 + mousedown/up', async () => {
await svc.click(mockPage, { x: 100, y: 200 }, 'full');
expect(mockCursor.moveTo).toHaveBeenCalledWith({ x: 100, y: 200 }, expect.any(Object));
expect(mockMouse.down).toHaveBeenCalled();
expect(mockMouse.up).toHaveBeenCalled();
});
it('fast 模式: 不走 ghost-cursor直接 mouse.move + 短延迟', async () => {
await svc.click(mockPage, { x: 100, y: 200 }, 'fast');
expect(mockCursor.moveTo).not.toHaveBeenCalled();
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
expect(mockMouse.down).toHaveBeenCalled();
});
it('off 模式: 立即 click无延迟', async () => {
(svc as any).rand = jest.fn();
await svc.click(mockPage, { x: 100, y: 200 }, 'off');
expect(mockCursor.moveTo).not.toHaveBeenCalled();
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
expect(mockMouse.down).toHaveBeenCalled();
expect((svc as any).rand).not.toHaveBeenCalled();
});
it('type full: 字符间随机间隔5% 概率错字mock 不触发)', async () => {
const keyboard = { type: jest.fn(), press: jest.fn() };
mockPage.keyboard = keyboard;
await svc.type(mockPage, 'hello', 'full');
// 'hello' 5 个字符 → keyboard.type 应该被调 5 次
expect(keyboard.type).toHaveBeenCalledTimes(5);
expect(keyboard.type).toHaveBeenNthCalledWith(1, 'h');
});
it('type off: 直接整体输入', async () => {
const keyboard = { type: jest.fn() };
mockPage.keyboard = keyboard;
await svc.type(mockPage, 'hello', 'off');
expect(keyboard.type).toHaveBeenCalledWith('hello', undefined);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest humanizer.service
```
- [ ] **Step 3: 实现 humanizer.service.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
import { createCursor } from 'ghost-cursor';
export type HumanizeMode = 'full' | 'fast' | 'off';
@Provide()
@Scope(ScopeEnum.Singleton)
export class HumanizerService {
// 注入点便于测试
protected createCursor = createCursor;
protected rand(min: number, max: number): number {
return min + Math.floor(Math.random() * (max - min));
}
protected chance(p: number): boolean {
return Math.random() < p;
}
async click(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
if (mode === 'full') {
// ghost-cursor 是 puppeteer 类型patchright Page 与之不兼容,需要 as any cast
const cursor = this.createCursor(page as any);
await cursor.moveTo(point, { randomizeMoveDelay: true, moveDelay: this.rand(50, 150) });
await this.sleep(this.rand(100, 300));
await page.mouse.down();
await this.sleep(this.rand(50, 150));
await page.mouse.up();
if (this.chance(0.2)) {
await page.mouse.wheel(0, this.rand(2, 10));
}
} else if (mode === 'fast') {
await page.mouse.move(point.x, point.y);
await this.sleep(this.rand(50, 200));
await page.mouse.down();
await this.sleep(this.rand(20, 50));
await page.mouse.up();
} else {
await page.mouse.move(point.x, point.y);
await page.mouse.down();
await page.mouse.up();
}
}
async type(page: any, text: string, mode: HumanizeMode): Promise<void> {
if (mode === 'off') {
await page.keyboard.type(text);
return;
}
const minDelay = mode === 'full' ? 80 : 30;
const maxDelay = mode === 'full' ? 250 : 80;
for (let i = 0; i < text.length; i++) {
const ch = text[i];
// 5% 错字(仅 full 模式)
if (mode === 'full' && this.chance(0.05)) {
const wrong = String.fromCharCode(ch.charCodeAt(0) + 1);
await page.keyboard.type(wrong);
await this.sleep(this.rand(minDelay, maxDelay));
await page.keyboard.press('Backspace');
await this.sleep(this.rand(minDelay, maxDelay));
}
await page.keyboard.type(ch);
await this.sleep(this.rand(minDelay, maxDelay));
}
}
async hover(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
if (mode === 'full') {
const cursor = this.createCursor(page as any);
await cursor.moveTo(point);
await this.sleep(this.rand(200, 800));
} else {
await page.mouse.move(point.x, point.y);
if (mode === 'fast') await this.sleep(this.rand(50, 200));
}
}
async scroll(page: any, deltaY: number, mode: HumanizeMode): Promise<void> {
if (mode === 'off') {
await page.mouse.wheel(0, deltaY);
return;
}
const steps = mode === 'full' ? Math.ceil(Math.abs(deltaY) / 50) : Math.ceil(Math.abs(deltaY) / 200);
const stepSize = deltaY / steps;
const minD = mode === 'full' ? 50 : 10;
const maxD = mode === 'full' ? 150 : 30;
for (let i = 0; i < steps; i++) {
await page.mouse.wheel(0, stepSize);
await this.sleep(this.rand(minD, maxD));
}
}
private sleep(ms: number): Promise<void> {
return new Promise(r => setTimeout(r, ms));
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest humanizer.service
```
预期PASS5 tests
- [ ] **Step 5: Checkpoint**
`Task 8 完成`humanizer 三档行为可用且可测。
---
## Task 9BrowserDaemonService.click/fill/type/snapshot/cookie/state业务方法
**Files:**
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts`
- [ ] **Step 1: 写 SnapshotRefService 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts
import { SnapshotRefService } from '../../../../src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.js';
describe('SnapshotRefService', () => {
let svc: SnapshotRefService;
let mockPage: any;
beforeEach(() => {
svc = new SnapshotRefService();
mockPage = {
evaluate: jest.fn().mockResolvedValue([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'input', text: '' },
]),
};
});
it('snapshot 调 page.evaluate 注入 data-ai-ref返回 refs', async () => {
const refs = await svc.snapshot('s1', mockPage);
expect(refs).toEqual([
{ ref: 'e1', tag: 'button', text: 'Login' },
{ ref: 'e2', tag: 'input', text: '' },
]);
expect(mockPage.evaluate).toHaveBeenCalledTimes(1);
});
it('refToSelector 返回 [data-ai-ref="..."]', () => {
expect(svc.refToSelector('e15')).toBe('[data-ai-ref="e15"]');
});
});
```
- [ ] **Step 2: 实现 SnapshotRefService**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
export interface RefEntry {
ref: string;
tag: string;
text: string;
}
@Provide()
@Scope(ScopeEnum.Singleton)
export class SnapshotRefService {
/** 给页面所有可交互元素注入 data-ai-ref返回 ref 列表 */
async snapshot(_sessionName: string, page: any): Promise<RefEntry[]> {
return page.evaluate(() => {
const interactive = Array.from(document.querySelectorAll(
'a, button, input, select, textarea, [role="button"], [role="link"], [contenteditable="true"]'
));
const out: { ref: string; tag: string; text: string }[] = [];
interactive.forEach((el, i) => {
const ref = `e${i + 1}`;
el.setAttribute('data-ai-ref', ref);
const tag = el.tagName.toLowerCase();
let text = (el as HTMLElement).innerText || (el as HTMLInputElement).value || '';
text = text.trim().slice(0, 80);
out.push({ ref, tag, text });
});
return out;
});
}
refToSelector(ref: string): string {
return `[data-ai-ref="${ref}"]`;
}
}
```
- [ ] **Step 3: 跑 snapshot-ref 测试**
```bash
pnpm jest snapshot-ref
```
预期PASS
- [ ] **Step 4: 在 BrowserDaemonService 中加业务方法**
`daemon.service.ts` 末尾加入(在 `setHumanizeMode` 之后):
```typescript
// ===== 业务方法(依赖 humanizer + snapshot-ref=====
@Inject() humanizer: HumanizerService;
@Inject() snapshotRef: SnapshotRefService;
async click(sessionName: string, ref: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const locator = page.locator(this.snapshotRef.refToSelector(ref));
const box = await locator.boundingBox();
if (!box) throw new Error(`Ref '${ref}' not visible`);
const m = mode ?? this.getHumanizeMode(sessionName);
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
});
}
async fill(sessionName: string, ref: string, value: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const locator = page.locator(this.snapshotRef.refToSelector(ref));
const box = await locator.boundingBox();
if (!box) throw new Error(`Ref '${ref}' not visible`);
const m = mode ?? this.getHumanizeMode(sessionName);
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
await locator.fill('');
await this.humanizer.type(page, value, m);
});
}
async type(sessionName: string, text: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
await this.humanizer.type(page, text, mode ?? this.getHumanizeMode(sessionName));
});
}
async snapshot(sessionName: string) {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
const refs = await this.snapshotRef.snapshot(sessionName, page);
return { url: page.url(), refs };
});
}
async getCookies(sessionName: string, domain?: string): Promise<any[]> {
return this.registry.withLock(sessionName, async () => {
const { ctx } = this.getPageOrFail(sessionName);
const cookies = await ctx.cookies();
return domain ? cookies.filter((c: any) => c.domain.includes(domain)) : cookies;
});
}
async saveState(sessionName: string, filePath: string): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { ctx } = this.getPageOrFail(sessionName);
await ctx.storageState({ path: filePath });
});
}
async loadState(sessionName: string, filePath: string): Promise<void> {
// loadState 需在 launchPersistentContext 时通过 storageState 选项注入
// 但 patchright launchPersistentContext 不直接接受 storageState
// → 实际方案addCookies + 手动 localStorage基于 file 内容)
return this.registry.withLock(sessionName, async () => {
const { ctx, page } = this.getPageOrFail(sessionName);
const fs = await import('node:fs/promises');
const data = JSON.parse(await fs.readFile(filePath, 'utf8'));
if (data.cookies) await ctx.addCookies(data.cookies);
if (data.origins) {
for (const o of data.origins) {
await page.goto(o.origin);
for (const item of o.localStorage ?? []) {
await page.evaluate(([k, v]: [string, string]) => localStorage.setItem(k, v), [item.name, item.value]);
}
}
}
});
}
async goto(sessionName: string, url: string): Promise<void> {
return this.registry.withLock(sessionName, async () => {
const { page } = this.getPageOrFail(sessionName);
await page.goto(url);
});
}
```
也要在文件顶部加 import
```typescript
import { HumanizerService } from './humanizer.service.js';
import { SnapshotRefService } from './snapshot-ref.service.js';
```
- [ ] **Step 5: 写新方法的测试(追加到 daemon.service.test.ts**
```typescript
// 追加到 daemon.service.test.ts
describe('BrowserDaemonService 业务方法', () => {
let svc: BrowserDaemonService;
let mockCtx: any;
let mockPage: any;
let mockHumanizer: any;
let mockSnapshotRef: any;
beforeEach(() => {
mockPage = {
url: () => 'https://example.com',
mouse: { down: jest.fn(), up: jest.fn(), move: jest.fn() },
keyboard: { type: jest.fn() },
locator: jest.fn().mockReturnValue({
boundingBox: jest.fn().mockResolvedValue({ x: 10, y: 20, width: 100, height: 40 }),
fill: jest.fn(),
}),
goto: jest.fn(),
};
mockCtx = {
pages: () => [mockPage],
cookies: jest.fn().mockResolvedValue([{ name: 'a', value: '1', domain: '.x.com' }]),
storageState: jest.fn(),
addCookies: jest.fn(),
};
mockHumanizer = { click: jest.fn(), type: jest.fn() };
mockSnapshotRef = {
refToSelector: (r: string) => `[data-ai-ref="${r}"]`,
snapshot: jest.fn().mockResolvedValue([{ ref: 'e1', tag: 'button', text: 'OK' }]),
};
svc = new BrowserDaemonService();
const { SessionRegistry } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js');
const { SessionScheduler } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js');
(svc as any).registry = new SessionRegistry();
(svc as any).scheduler = new SessionScheduler({ maxActiveSessions: 5 });
(svc as any).humanizer = mockHumanizer;
(svc as any).snapshotRef = mockSnapshotRef;
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
(svc as any).registry.register('s1', mockCtx);
(svc as any).scheduler.recordActivate('s1');
});
it('click: 调 humanizer.click(中心点, mode)', async () => {
await svc.click('s1', 'e1');
expect(mockHumanizer.click).toHaveBeenCalledWith(mockPage, { x: 60, y: 40 }, 'full');
});
it('snapshot: 调 snapshotRef + 返回 url + refs', async () => {
const r = await svc.snapshot('s1');
expect(r.url).toBe('https://example.com');
expect(r.refs).toEqual([{ ref: 'e1', tag: 'button', text: 'OK' }]);
});
it('getCookies: domain 过滤', async () => {
const r = await svc.getCookies('s1', 'x.com');
expect(r).toEqual([{ name: 'a', value: '1', domain: '.x.com' }]);
});
it('saveState: 调 storageState({path})', async () => {
await svc.saveState('s1', '/tmp/s.json');
expect(mockCtx.storageState).toHaveBeenCalledWith({ path: '/tmp/s.json' });
});
});
```
- [ ] **Step 6: 跑所有 daemon 测试**
```bash
pnpm jest daemon.service
```
预期PASS前 5 tests + 4 新 tests = 9
- [ ] **Step 7: Checkpoint**
`Task 9 完成`BrowserDaemonService 业务方法齐备。
---
## Task 9bCleanup + Lifecycle + Idle-Timeout**新增 P1 修复**
**问题**spec §5.4 要求 @Init 扫 known sessions、@Destroy 优雅关闭 + 5s 超时强制 kill§6.6 要求 60min idle 自动回收。原 plan 未实现。
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts`
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts`
- [ ] **Step 1: 实现 cleanup.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts
/**
* 优雅关闭工具:等待 ctx.close() 最多 5s超时强制 kill
*/
export async function gracefullyClose(ctx: any, timeoutMs = 5000): Promise<void> {
await Promise.race([
ctx.close().catch(() => {}),
new Promise(resolve => setTimeout(resolve, timeoutMs)),
]);
// 强制 kill 残留 chrome 进程
try {
const browser = ctx.browser?.();
if (browser?.process) {
const proc = browser.process();
if (proc && !proc.killed) proc.kill('SIGKILL');
}
} catch {}
}
```
- [ ] **Step 2: SessionScheduler 增加 idle 计时器**
修改 `session-scheduler.ts`,加入:
```typescript
// 在 SessionScheduler 类内
private idleTimers = new Map<string, NodeJS.Timeout>();
private idleTimeoutMs: number;
private onIdleTimeout?: (sessionName: string) => Promise<void>;
constructor(opts: SchedulerOptions = {}) {
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
this.idleTimeoutMs = opts.idleTimeoutMs ?? 60 * 60 * 1000; // 60 min
}
setIdleTimeoutHandler(fn: (sessionName: string) => Promise<void>): void {
this.onIdleTimeout = fn;
}
touch(sessionName: string): void {
const s = this.sessions.get(sessionName);
if (s) {
s.status = 'active';
s.lastUsedAt = new Date();
}
this.resetIdleTimer(sessionName);
}
private resetIdleTimer(sessionName: string): void {
this.clearIdleTimer(sessionName);
if (this.idleTimeoutMs <= 0 || !this.onIdleTimeout) return;
const timer = setTimeout(() => {
this.idleTimers.delete(sessionName);
this.onIdleTimeout?.(sessionName).catch(() => {});
}, this.idleTimeoutMs);
this.idleTimers.set(sessionName, timer);
}
private clearIdleTimer(sessionName: string): void {
const t = this.idleTimers.get(sessionName);
if (t) {
clearTimeout(t);
this.idleTimers.delete(sessionName);
}
}
// 在 recordActivate 末尾调 resetIdleTimer
// 在 recordDeactivate 调 clearIdleTimer
```
- [ ] **Step 3: daemon.service @Init / @Destroy 钩子**
`daemon.service.ts` 加:
```typescript
import { Init, Destroy } from '@midwayjs/core';
import { gracefullyClose } from '../runtime/cleanup.js';
import { getStateDir } from '../runtime/browser-data-dir.js';
import * as path from 'node:path';
import * as fs from 'node:fs';
// 在 BrowserDaemonService 类内:
@Init()
async init(): Promise<void> {
// 注册 idle 回收回调
this.scheduler.setIdleTimeoutHandler(async (sessionName) => {
this.logger.info(`[browser-daemon] idle timeout: ${sessionName} → save+close`);
await this.registry.withLock(sessionName, async () => {
const ctx = this.registry.get(sessionName);
if (!ctx) return;
const stateFile = getStateDir(sessionName);
try {
await ctx.storageState({ path: stateFile });
} catch (e: any) {
this.logger.warn(`[browser-daemon] saveState on idle failed: ${e.message}`);
}
await this.closeInternal(sessionName);
});
});
// 扫描 known sessions不主动 launch
const stateRoot = path.dirname(getStateDir('placeholder'));
if (fs.existsSync(stateRoot)) {
const files = fs.readdirSync(stateRoot).filter(f => f.endsWith('.json'));
this.logger.info(`[browser-daemon] init: ${files.length} known sessions on disk`);
}
}
@Destroy()
async destroy(): Promise<void> {
this.logger.info('[browser-daemon] destroying, closing all sessions...');
const all = this.registry.list();
await Promise.all(all.map(async (e) => {
try {
const stateFile = getStateDir(e.sessionName);
await e.context.storageState({ path: stateFile });
} catch (err: any) {
this.logger.warn(`[browser-daemon] saveState ${e.sessionName} failed: ${err.message}`);
}
await gracefullyClose(e.context, 5000);
}));
}
```
- [ ] **Step 4: 写 cleanup 测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts
import { gracefullyClose } from '../../../../src/modules/netaclaw/browser-daemon/runtime/cleanup.js';
describe('cleanup.gracefullyClose', () => {
it('正常关闭close 完成', async () => {
const close = jest.fn().mockResolvedValue(undefined);
await gracefullyClose({ close }, 1000);
expect(close).toHaveBeenCalled();
});
it('close 超时5s 后强制 kill', async () => {
const kill = jest.fn();
const ctx = {
close: () => new Promise(() => {}), // 永远不 resolve
browser: () => ({ process: () => ({ kill, killed: false }) }),
};
const start = Date.now();
await gracefullyClose(ctx, 100);
expect(Date.now() - start).toBeGreaterThanOrEqual(100);
expect(kill).toHaveBeenCalledWith('SIGKILL');
});
it('close 抛错不阻断', async () => {
await expect(
gracefullyClose({ close: () => Promise.reject(new Error('boom')) }, 100)
).resolves.toBeUndefined();
});
});
```
- [ ] **Step 5: 跑测试**
```bash
pnpm jest cleanup.test
```
预期PASS3 tests
- [ ] **Step 6: SessionScheduler idle timer 测试追加**
```typescript
// 追加到 session-scheduler.test.ts
describe('SessionScheduler idle timer', () => {
it('idle 超时后调 onIdleTimeout', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 50 });
const handler = jest.fn().mockResolvedValue(undefined);
sch.setIdleTimeoutHandler(handler);
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 100));
expect(handler).toHaveBeenCalledWith('s1');
});
it('touch 重置计时器', async () => {
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 80 });
const handler = jest.fn().mockResolvedValue(undefined);
sch.setIdleTimeoutHandler(handler);
sch.recordActivate('s1');
await new Promise(r => setTimeout(r, 50));
sch.touch('s1');
await new Promise(r => setTimeout(r, 50));
// 累积 100ms 但 touch 重置后只过了 50ms未触发
expect(handler).not.toHaveBeenCalled();
await new Promise(r => setTimeout(r, 50));
expect(handler).toHaveBeenCalledWith('s1');
});
});
```
- [ ] **Step 7: Checkpoint**
`Task 9b 完成`lifecycle 钩子 + idle 自动回收 + 优雅关闭。验收 9 + 5b 可通过。
---
## Task 10control-auth.middlewareloopback + secret
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts`
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts`
- [ ] **Step 1: 写测试**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts
import { BrowserControlAuthMiddleware } from '../../../../src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.js';
describe('BrowserControlAuthMiddleware', () => {
const mw = new BrowserControlAuthMiddleware();
const fn = mw.resolve();
beforeEach(() => {
process.env.NETA_TRAY_SECRET = 'test-secret';
process.env.NODE_ENV = 'test';
});
it('loopback + 正确 secret → 通过', async () => {
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(next).toHaveBeenCalled();
});
it('非 loopback → 403', async () => {
const ctx: any = { ip: '8.8.8.8', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(403);
expect(next).not.toHaveBeenCalled();
});
it('loopback + 错误 secret → 401', async () => {
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'wrong' }, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(401);
});
it('loopback 无 secret → 401', async () => {
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(401);
});
it('dev 模式 + secret 未配置 → 旁路通过loopback 仍校验)', async () => {
process.env.NETA_TRAY_SECRET = '';
process.env.NODE_ENV = 'development';
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(next).toHaveBeenCalled();
});
it('dev 模式 + secret 未配置 + 非 loopback → 仍 403', async () => {
process.env.NETA_TRAY_SECRET = '';
process.env.NODE_ENV = 'development';
const ctx: any = { ip: '8.8.8.8', headers: {}, status: 200 };
const next = jest.fn();
await fn(ctx, next);
expect(ctx.status).toBe(403);
});
});
```
- [ ] **Step 2: 跑测试看失败**
```bash
pnpm jest control-auth.middleware
```
- [ ] **Step 3: 实现**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts
import { IMiddleware, Middleware } from '@midwayjs/core';
import { NextFunction, Context } from '@midwayjs/koa';
import { isLoopbackAddress, validateRuntimeSecret } from '../../../../comm/runtime-secret.js';
@Middleware()
export class BrowserControlAuthMiddleware implements IMiddleware<Context, NextFunction> {
resolve() {
return async (ctx: Context, next: NextFunction) => {
const ip = ctx.ip || (ctx as any).request?.ip || (ctx as any).req?.socket?.remoteAddress;
if (!isLoopbackAddress(ip)) {
ctx.status = 403;
ctx.body = { code: 1003, error: 'Forbidden: only loopback access allowed' };
return;
}
const expected = process.env.NETA_TRAY_SECRET ?? '';
const actual = ctx.headers['x-neta-control-secret'] as string | undefined;
const isDev = process.env.NODE_ENV !== 'production';
// dev 旁路secret 未配置时仅信任 loopback
if (isDev && !expected) {
await next();
return;
}
if (!validateRuntimeSecret(expected, actual)) {
ctx.status = 401;
ctx.body = { code: 1001, error: 'Unauthorized: invalid x-neta-control-secret' };
return;
}
await next();
};
}
match(ctx: Context): boolean {
return ctx.path?.startsWith('/admin/browser-daemon/');
}
static getName(): string {
return 'browserControlAuth';
}
}
```
- [ ] **Step 4: 跑测试看通过**
```bash
pnpm jest control-auth.middleware
```
预期PASS6 tests
- [ ] **Step 5: 在 netaclaw/config.ts 注册中间件(重要:模块挂载明确)**
修改 `packages/backend/src/modules/netaclaw/config.ts`,把空的 `middlewares: []` 改为:
```typescript
import { ModuleConfig } from '@cool-midway/core';
import { BrowserControlAuthMiddleware } from './browser-daemon/middleware/control-auth.middleware.js';
export default () => {
return {
name: 'NetaClaw',
description: 'NetaClaw 电商浏览器自动化 Agent 引擎',
middlewares: [BrowserControlAuthMiddleware],
globalMiddlewares: [],
order: 0,
} as ModuleConfig;
};
```
> 关键browser-daemon **不是独立模块**,是 netaclaw 的子目录。中间件挂载到 netaclaw 模块,靠 `match()` 限制只在 `/admin/browser-daemon/*` 路径生效。其他 netaclaw 路由(`/admin/netaclaw/*`)不受影响。
注意保留 NetaClawConfig / defaultNetaClawConfig 等导出(在 `// ---` 分隔符之后的部分)。
- [ ] **Step 6: Checkpoint**
`Task 10 完成`auth middleware 校验 loopback + secret含 dev 旁路middleware 已挂载到 netaclaw 模块。
---
## Task 11HTTP Controllers5 个 controller
**Files:**
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts`
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts`
> 这 5 个 controller 都是 thin wrapper解析 body → 调 service → 返回结果。**严格按 Neta 现有 controller 风格**
> - `@Provide()` + `@Controller('/admin/browser-daemon')` 类装饰器声明路径前缀
> - 方法用相对路径 `@Post('/open')` `@Get('/list')`
> - 直接返回 `{ code: 1000, data: ... }`,不继承 `BaseController`,不用 `this.ok()`,不用 `@CoolController` 不用 `@CoolTag`
> - 参考 `packages/backend/src/modules/netaclaw/controller/admin/agent_channel.ts` 实际风格
- [ ] **Step 1: 实现 session.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts
import { Provide, Inject, Controller, Post, Get, Body } from '@midwayjs/core';
import { Context } from '@midwayjs/koa';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonSessionController {
@Inject() daemonService: BrowserDaemonService;
@Inject() ctx: Context;
@Post('/open')
async open(@Body() dto: any) {
try {
const r = await this.daemonService.open(dto);
return { code: 1000, data: r };
} catch (e: any) {
this.ctx.status = e.statusCode ?? 400;
return { code: 1002, error: e.message, retryAfter: e.retryAfter };
}
}
@Post('/close')
async close(@Body('sessionName') sessionName: string) {
await this.daemonService.close(sessionName);
return { code: 1000, data: { ok: true } };
}
@Get('/list')
async list() {
return { code: 1000, data: this.daemonService.list() };
}
@Get('/stats')
async stats() {
return { code: 1000, data: this.daemonService.getStats() };
}
}
```
- [ ] **Step 2: 实现 interaction.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonInteractionController {
@Inject() daemonService: BrowserDaemonService;
@Post('/click')
async click(@Body() dto: { sessionName: string; ref: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.click(dto.sessionName, dto.ref, dto.mode ?? 'full');
return { code: 1000, data: { clicked: dto.ref } };
}
@Post('/fill')
async fill(@Body() dto: { sessionName: string; ref: string; value: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.fill(dto.sessionName, dto.ref, dto.value, dto.mode ?? 'full');
return { code: 1000, data: { filled: dto.ref } };
}
@Post('/type')
async type(@Body() dto: { sessionName: string; text: string; mode?: 'full' | 'fast' | 'off' }) {
await this.daemonService.type(dto.sessionName, dto.text, dto.mode ?? 'full');
return { code: 1000, data: { typed: dto.text.length } };
}
}
```
- [ ] **Step 3: 实现 navigation.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonNavigationController {
@Inject() daemonService: BrowserDaemonService;
@Post('/goto')
async goto(@Body() dto: { sessionName: string; url: string }) {
await this.daemonService.goto(dto.sessionName, dto.url);
return { code: 1000, data: { navigated: dto.url } };
}
}
```
- [ ] **Step 4: 实现 state.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonStateController {
@Inject() daemonService: BrowserDaemonService;
@Post('/cookie-list')
async cookieList(@Body() dto: { sessionName: string; domain?: string }) {
return { code: 1000, data: await this.daemonService.getCookies(dto.sessionName, dto.domain) };
}
@Post('/state-save')
async stateSave(@Body() dto: { sessionName: string; filePath: string }) {
await this.daemonService.saveState(dto.sessionName, dto.filePath);
return { code: 1000, data: { saved: dto.filePath } };
}
@Post('/state-load')
async stateLoad(@Body() dto: { sessionName: string; filePath: string }) {
await this.daemonService.loadState(dto.sessionName, dto.filePath);
return { code: 1000, data: { loaded: dto.filePath } };
}
}
```
- [ ] **Step 5: 实现 inspect.ts**
```typescript
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
import { BrowserDaemonService } from '../../service/daemon.service.js';
@Provide()
@Controller('/admin/browser-daemon')
export class AdminBrowserDaemonInspectController {
@Inject() daemonService: BrowserDaemonService;
@Post('/snapshot')
async snapshot(@Body() dto: { sessionName: string }) {
return { code: 1000, data: await this.daemonService.snapshot(dto.sessionName) };
}
}
```
- [ ] **Step 6: 启动 backend 验证路由**
dev 模式启动dev 旁路时无需 secret
```bash
cd packages/backend
NODE_ENV=development pnpm dev
```
另一终端:
```bash
# dev 旁路middleware 已实现):仅 loopback 即可secret 未设也能调
curl -X GET http://localhost:8003/admin/browser-daemon/list
```
预期200 + `{"code":1000,"data":[]}`
- [ ] **Step 7: Checkpoint**
`Task 11 完成`5 个 controller 上线HTTP API 可达。
---
## Task 12Contract TestHTTP ↔ service 等价性)
**Files:**
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts`
- [ ] **Step 1: 写 contract test**
```typescript
// packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts
import { createApp, close } from '@midwayjs/mock';
import { Framework } from '@midwayjs/koa';
describe('BrowserDaemon HTTP ↔ Service 契约', () => {
let app: any;
beforeAll(async () => {
process.env.NETA_TRAY_SECRET = 'ct-secret';
app = await createApp(undefined, undefined, Framework);
});
afterAll(async () => { if (app) await close(app); });
/** Mock daemonService验证 controller 调它的方法名/参数与 HTTP 1:1 */
const cases = [
{ http: { method: 'POST', path: '/admin/browser-daemon/close', body: { sessionName: 's1' } }, service: 'close', args: ['s1'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/click', body: { sessionName: 's1', ref: 'e1', mode: 'full' } }, service: 'click', args: ['s1', 'e1', 'full'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/fill', body: { sessionName: 's1', ref: 'e1', value: 'abc' } }, service: 'fill', args: ['s1', 'e1', 'abc', undefined] },
{ http: { method: 'POST', path: '/admin/browser-daemon/cookie-list', body: { sessionName: 's1', domain: 'x.com' } }, service: 'getCookies', args: ['s1', 'x.com'] },
{ http: { method: 'POST', path: '/admin/browser-daemon/snapshot', body: { sessionName: 's1' } }, service: 'snapshot', args: ['s1'] },
];
for (const c of cases) {
it(`HTTP ${c.http.method} ${c.http.path} → service.${c.service}(${c.args.join(',')})`, async () => {
const fakeService = {
[c.service]: jest.fn().mockResolvedValue(undefined),
};
// 替换 daemonService 实例
const container = (app as any).getApplicationContext();
container.registerObject('browserDaemonService', fakeService);
// 调 HTTP
const supertest = require('supertest');
const res = await supertest(app.getServer())[c.http.method.toLowerCase()](c.http.path)
.set('x-neta-control-secret', 'ct-secret')
.send(c.http.body);
expect(res.status).toBe(200);
expect(fakeService[c.service]).toHaveBeenCalledWith(...c.args);
});
}
});
```
> 注:实施时依据 Midway test 实际 API 调整 `registerObject` / `supertest` 接入;目的是验证 HTTP 与 service 一一对应。
- [ ] **Step 2: 跑 contract test**
```bash
pnpm jest contract.test
```
预期PASS5 cases
- [ ] **Step 3: Checkpoint**
`Task 12 完成`HTTP 与 service 等价性有 contract test 兜底。
---
## Task 13-18netabrowser-cli 包实现(合并描述)
> 这 6 个 task 是 CLI 包内部各文件,模式相似(接受参数 → 调 HTTP → 输出)。每个文件按下面同一套结构实现,不再展开每 step。
### Task 13bin/main.tscommander 解析 + 命令分发)
**Files:**
- Create: `packages/netabrowser-cli/src/bin/main.ts`
- [ ] **Step 1: 实现**
```typescript
#!/usr/bin/env node
// packages/netabrowser-cli/src/bin/main.ts
import { Command } from 'commander';
import { registerSessionCommands } from '../commands/session.js';
import { registerInteractionCommands } from '../commands/interaction.js';
import { registerNavigationCommands } from '../commands/navigation.js';
import { registerStateCommands } from '../commands/state.js';
import { registerInspectCommands } from '../commands/inspect.js';
const program = new Command();
program
.name('netabrowser-cli')
.description('Neta 反风控+拟人化浏览器 CLI')
.version('0.1.0');
registerSessionCommands(program);
registerInteractionCommands(program);
registerNavigationCommands(program);
registerStateCommands(program);
registerInspectCommands(program);
program.parseAsync(process.argv).catch(e => {
console.error('Error:', e.message);
process.exit(1);
});
```
- [ ] **Step 2: Checkpoint**
### Task 14client/runtime-info + http-client
**Files:**
- Create: `packages/netabrowser-cli/src/client/runtime-info.ts`
- Create: `packages/netabrowser-cli/src/client/http-client.ts`
- [ ] **Step 1: runtime-info.ts**
```typescript
// packages/netabrowser-cli/src/client/runtime-info.ts
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
export interface RuntimeInfoLite {
port: number;
controlSecret: string;
controlBaseUrl: string;
}
/**
* v2 修复 P1-6CLI 启动时 cwd 不一定 = backend cwd。
* runtime-info.json 真实写入 packages/backend/dist/runtime-info.json。
* 优先级:环境变量 > pkg 同目录 > monorepo backend dist > user home。
*/
export function readRuntimeInfo(): RuntimeInfoLite {
if (process.env.NETA_RUNTIME_INFO) {
return JSON.parse(fs.readFileSync(process.env.NETA_RUNTIME_INFO, 'utf8'));
}
const candidates = [
// pkg 模式netabrowser-cli.exe 同目录 data/runtime-info.json
path.join(path.dirname(process.execPath), 'data', 'runtime-info.json'),
// dev 模式:从 cli 编译产物上溯找 monorepo再进 backend dist
// __dirname = <root>/packages/netabrowser-cli/dist/client
// 上溯 4 层到 monorepo root再进 packages/backend/dist
path.resolve(__dirname, '../../../../../packages/backend/dist/runtime-info.json'),
// 兜底:用户主目录
path.join(os.homedir(), '.neta', 'runtime-info.json'),
];
for (const p of candidates) {
if (fs.existsSync(p)) return JSON.parse(fs.readFileSync(p, 'utf8'));
}
throw new Error(
`runtime-info.json not found. Set NETA_RUNTIME_INFO env var or run backend first.\nTried:\n${candidates.map(c => ' - ' + c).join('\n')}`
);
}
```
- [ ] **Step 2: http-client.ts**
```typescript
// packages/netabrowser-cli/src/client/http-client.ts
import axios, { AxiosInstance } from 'axios';
import { readRuntimeInfo } from './runtime-info.js';
let client: AxiosInstance | null = null;
export function getHttpClient(): AxiosInstance {
if (client) return client;
const info = readRuntimeInfo();
client = axios.create({
baseURL: info.controlBaseUrl ?? `http://127.0.0.1:${info.port}`,
headers: { 'x-neta-control-secret': info.controlSecret },
timeout: 60_000,
});
return client;
}
export async function callDaemon<T = any>(path: string, body?: any): Promise<T> {
const c = getHttpClient();
const res = await c.post(path, body ?? {});
if (res.data?.code !== 1000) {
throw new Error(`Daemon error: ${res.data?.message ?? 'unknown'}`);
}
return res.data.data as T;
}
```
- [ ] **Step 3: Checkpoint**
### Task 15commands/session.tsopen/close/list
```typescript
// packages/netabrowser-cli/src/commands/session.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerSessionCommands(program: Command) {
program
.command('open <url>')
.description('启动会话并打开 URL')
.requiredOption('--session <name>', 'session 名称')
.option('--proxy <url>', '代理 URLhttp://user:pass@host:port')
.option('--fingerprint-seed <n>', '指纹 seed', parseInt)
.option('--profile-dir <name>', 'profile 目录名')
.option('--headed', '有头模式')
.option('--humanize-mode <mode>', 'full|fast|off', 'full')
.option('--priority <p>', 'low|normal|high', 'normal')
.option('--queue', '排队等位')
.option('--raw', '输出 raw JSON')
.action(async (url, opts) => {
const dto: any = {
sessionName: opts.session,
url,
humanizeMode: opts.humanizeMode,
priority: opts.priority,
queue: !!opts.queue,
headed: !!opts.headed,
};
if (opts.fingerprintSeed != null) dto.fingerprintSeed = opts.fingerprintSeed;
if (opts.profileDir) dto.profileDir = opts.profileDir;
if (opts.proxy) dto.proxy = parseProxyUrl(opts.proxy);
const r = await callDaemon('/admin/browser-daemon/open', dto);
console.log(formatOutput(r, opts.raw));
});
program
.command('close')
.requiredOption('--session <name>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/close', { sessionName: opts.session });
console.log(formatOutput({ ok: true }, opts.raw));
});
program
.command('list')
.description('列出所有会话')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/list');
console.log(formatOutput(r, opts.raw));
});
}
function parseProxyUrl(url: string) {
const u = new URL(url);
return {
server: `${u.protocol}//${u.host}`,
username: decodeURIComponent(u.username) || undefined,
password: decodeURIComponent(u.password) || undefined,
};
}
```
- [ ] **Checkpoint**
### Task 16commands/interaction.ts
```typescript
// packages/netabrowser-cli/src/commands/interaction.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerInteractionCommands(program: Command) {
program
.command('click <ref>')
.requiredOption('--session <name>')
.option('--mode <mode>', 'full|fast|off')
.option('--raw')
.action(async (ref, opts) => {
// v2 修复 P1-8CLI 端 mode 优先级 = 命令级 --mode > NETA_BROWSER_HUMANIZE_MODE 环境变量 > 默认 full
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/click', { sessionName: opts.session, ref, mode });
console.log(formatOutput({ clicked: ref }, opts.raw));
});
program
.command('fill <ref> <value>')
.requiredOption('--session <name>')
.option('--mode <mode>')
.option('--raw')
.action(async (ref, value, opts) => {
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/fill', { sessionName: opts.session, ref, value, mode });
console.log(formatOutput({ filled: ref }, opts.raw));
});
program
.command('type <text>')
.requiredOption('--session <name>')
.option('--mode <mode>')
.option('--raw')
.action(async (text, opts) => {
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
await callDaemon('/admin/browser-daemon/type', { sessionName: opts.session, text, mode });
console.log(formatOutput({ typed: text.length }, opts.raw));
});
}
```
- [ ] **Checkpoint**
### Task 17commands/navigation + state + inspect
```typescript
// packages/netabrowser-cli/src/commands/navigation.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerNavigationCommands(program: Command) {
program
.command('goto <url>')
.requiredOption('--session <name>')
.option('--raw')
.action(async (url, opts) => {
await callDaemon('/admin/browser-daemon/goto', { sessionName: opts.session, url });
console.log(formatOutput({ navigated: url }, opts.raw));
});
}
```
```typescript
// packages/netabrowser-cli/src/commands/state.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerStateCommands(program: Command) {
program
.command('cookie-list')
.requiredOption('--session <name>')
.option('--domain <d>')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/cookie-list', { sessionName: opts.session, domain: opts.domain });
console.log(formatOutput(r, opts.raw));
});
program
.command('state-save')
.requiredOption('--session <name>')
.requiredOption('--output <path>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/state-save', { sessionName: opts.session, filePath: opts.output });
console.log(formatOutput({ saved: opts.output }, opts.raw));
});
program
.command('state-load')
.requiredOption('--session <name>')
.requiredOption('--input <path>')
.option('--raw')
.action(async (opts) => {
await callDaemon('/admin/browser-daemon/state-load', { sessionName: opts.session, filePath: opts.input });
console.log(formatOutput({ loaded: opts.input }, opts.raw));
});
}
```
```typescript
// packages/netabrowser-cli/src/commands/inspect.ts
import { Command } from 'commander';
import { callDaemon } from '../client/http-client.js';
import { formatOutput } from '../output/formatter.js';
export function registerInspectCommands(program: Command) {
program
.command('snapshot')
.requiredOption('--session <name>')
.option('--raw')
.action(async (opts) => {
const r = await callDaemon('/admin/browser-daemon/snapshot', { sessionName: opts.session });
console.log(formatOutput(r, opts.raw));
});
}
```
- [ ] **Checkpoint**
### Task 18output/formatter.ts
```typescript
// packages/netabrowser-cli/src/output/formatter.ts
export function formatOutput(data: any, raw?: boolean): string {
if (raw) return JSON.stringify(data);
if (Array.isArray(data) && data.length > 0 && data[0]?.ref) {
// snapshot 友好输出
return data.map((r: any) => `[${r.ref}] ${r.tag}: ${r.text}`).join('\n');
}
return JSON.stringify(data, null, 2);
}
```
写最小测试:
```typescript
// packages/netabrowser-cli/tests/output-formatter.test.ts
import { formatOutput } from '../src/output/formatter.js';
describe('formatter', () => {
it('--raw → 单行 JSON', () => {
expect(formatOutput({ a: 1 }, true)).toBe('{"a":1}');
});
it('snapshot refs → 友好输出', () => {
const r = formatOutput([{ ref: 'e1', tag: 'button', text: 'OK' }]);
expect(r).toContain('[e1] button: OK');
});
});
```
- [ ] **StepCheckpoint**
---
## Task 19SKILL.mdNetaClaw Agent skill 元数据)
**Files:**
- Create: `packages/backend/skills/netabrowser-cli/SKILL.md`
```markdown
---
name: netabrowser-cli
description: Neta 反风控+拟人化浏览器自动化 CLI。养号、电商自动化、AI 探索复杂网页时使用。比 playwright-cli 更难被反风控识别fingerprint-chromium 内核 + patchright 反自动化 + ghost-cursor 拟人化轨迹。100 账号矩阵养号场景首选。
allowed-tools: Bash(netabrowser-cli:*) Bash(npx:*)
---
# Browser Automation with netabrowser-cli
## When to use vs playwright-cli vs patchwright-cli
- **netabrowser-cli**:国内反风控严的站(小红书/抖音/淘宝/拼多多/微博)+ 多账号矩阵 + 养号场景
- **patchwright-cli**:国外 Cloudflare/DataDome 等反爬保护的站
- **playwright-cli**:测试、无反风控的站
If targeting Chinese social/e-commerce sites or running multi-account automation, **prefer netabrowser-cli**.
## Quick start
```bash
# 启动会话(每会话独立指纹+独立 IP
netabrowser-cli open https://www.xiaohongshu.com \
--session=acc1 \
--proxy=http://user:pass@host:port \
--fingerprint-seed=12345 \
--headed
# 拿当前页面可交互元素
netabrowser-cli snapshot --session=acc1
# 输出:[e1] button: 登录 / [e2] input: 手机号 / ...
# 拟人化点击 + 输入(默认 humanize-mode=full
netabrowser-cli click e1 --session=acc1
netabrowser-cli fill e2 13800001234 --session=acc1
# 抓 cookie
netabrowser-cli cookie-list --session=acc1 --domain=xiaohongshu.com
# 保存登录态
netabrowser-cli state-save --session=acc1 --output=/path/to/state.json
# 关闭
netabrowser-cli close --session=acc1
```
## Commands
### Session 管理
- `open <url>` 启动会话
- `close` 关闭
- `list` 列出所有 session
### 交互(默认拟人化 full
- `click <ref>` 点击 ref 元素(贝塞尔轨迹+视觉停顿+mousedown/up
- `fill <ref> <value>` focus + 拟人化输入
- `type <text>` 在已 focused 输入框打字
- `--mode=fast` 单命令切批量模式200-500ms 延迟,无轨迹)
- `--mode=off` 测试用,立即执行
### 导航
- `goto <url>` 跳转
### 状态
- `cookie-list [--domain=x]`
- `state-save --output=path` 保存完整登录态cookie+localStorage
- `state-load --input=path` 恢复
### 检查
- `snapshot` 拿可交互元素 ref 列表
## 参数规则
### 指纹
- `--fingerprint-seed=N` 单一 seed 派生所有指纹维度(推荐)
- 不同 seed → 不同 canvas/webgl/UA fingerprint同 seed → 完全可复现
### 代理
- `--proxy=http://user:pass@host:port` 单参数最方便
- 出口 IP 实测 = 代理 IPpatchright `launchPersistentContext` 验证通过)
### 拟人化档位
| `--humanize-mode` | 单命令开销 | 适用 |
|---|---|---|
| full默认 | 2-5s | 养号、敏感操作、AI 探索 |
| fast | 0.3-0.7s | 批量发布、批量评论 |
| off | <100ms | 测试CI |
### 调度
- `--priority=low\|normal\|high` 优先级
- `--queue` 触达上限时排队(最长 30s
## 性能特征(重要)
养号场景每号约 30 操作 × 5s ≈ 2.5min。100 号串行 ≈ 4 小时。批量发布请用 `--humanize-mode=fast`
## 详细文档
- [拟人化档位详解](references/humanization.md)
- [指纹参数](references/fingerprint.md)
- [代理配置](references/proxy.md)
- [典型场景示例](references/examples.md)
```
- [ ] **StepCheckpoint**
## Task 20references/ 子文档
> 创建 4 个 reference 文件作为 SKILL.md 的扩展。每个文件 50-100 行,按 SKILL.md 中链接对应的主题展开。
**Files:**
- Create: `packages/backend/skills/netabrowser-cli/references/humanization.md`
- Create: `packages/backend/skills/netabrowser-cli/references/fingerprint.md`
- Create: `packages/backend/skills/netabrowser-cli/references/proxy.md`
- Create: `packages/backend/skills/netabrowser-cli/references/examples.md`
每个文件简明阐述对应主题,含示例命令。具体内容由实施者从 spec §6.1-6.6 + 实测脚本提炼。
- [ ] **StepCheckpoint**
---
## Task 21Windows 打包集成
**Files:**
- Modify: `packages/backend/scripts/pkg-build.js`
- Modify: `packages/backend/scripts/build-windows-installer.js`
- Modify: `packages/backend/installer/setup.iss`
- [ ] **Step 1: pkg-build.js 增加 netabrowser-cli 编译**
`pkg-build.js` 现有逻辑,在 backend.exe 编译之后增加:
```javascript
// 编译 netabrowser-cli
console.log('Building netabrowser-cli...');
const cliRoot = path.resolve(__dirname, '../../netabrowser-cli');
execSync('pnpm build', { cwd: cliRoot, stdio: 'inherit' });
// pkg 打包
const pkgConfig = {
name: 'netabrowser-cli',
bin: path.join(cliRoot, 'dist', 'bin', 'main.js'),
targets: ['node22-win-x64'],
outputPath: path.join(__dirname, '../installer/dist/netabrowser-cli.exe'),
};
await pkg(pkgConfig);
```
- [ ] **Step 2: setup.iss 增加 [Files] 段**
在现有 `[Files]` 后追加:
```iss
Source: "dist/netabrowser-cli.exe"; DestDir: "{app}"; Flags: ignoreversion
Source: "..\..\netabrowser-cli\chromium\win64\*"; DestDir: "{app}\chromium\win64"; Flags: ignoreversion recursesubdirs createallsubdirs
Source: "..\..\backend\skills\netabrowser-cli\*"; DestDir: "{app}\skills\netabrowser-cli"; Flags: ignoreversion recursesubdirs createallsubdirs
```
- [ ] **Step 3: build-windows-installer.js 加 chromium 复制步骤**
在调 ISCC 之前确保 `chromium/win64` 存在。
- [ ] **Step 4: 实跑一次安装包构建dev 机器)**
```bash
cd packages/backend
pnpm run build:windows-installer
```
预期:生成的 setup.exe 大小 ~250-400MB。装到测试 Windows能找到 `C:\Program Files\Neta\netabrowser-cli.exe``chromium/win64/chrome.exe`
- [ ] **Step 5: Checkpoint**
---
## Task 22联调与冒烟手工验收
按 spec §9 验收清单逐条跑:
- [ ] **Step 1: 跑所有单测**
```bash
cd packages/backend
pnpm jest test/modules/netaclaw/browser-daemon/
```
预期:全 PASS
- [ ] **Step 2: 启动 backend dev 模式**
```bash
NODE_ENV=development pnpm dev
```
预期runtime-info.json 写入 `packages/backend/dist/runtime-info.json` 含 controlSecretdev 模式下 secret 可能为空字符串middleware 已开 dev 旁路)
> v2 注dev 模式下 NETA_TRAY_SECRET 通常未设。middleware 已实现 dev 旁路:`NODE_ENV !== 'production' && expected === ''` → 仅校验 loopback、跳过 secret 校验。
- [ ] **Step 3: cli list 应返回空**
```bash
node packages/netabrowser-cli/dist/bin/main.js list --raw
```
预期:`[]`
- [ ] **Step 4: 真代理出口 IP 验证**
```bash
node packages/netabrowser-cli/dist/bin/main.js open https://httpbin.org/ip \
--session=test \
--proxy="http://e50b26:7ecdccfd@210.51.27.112:10000" \
--headed
```
然后:
```bash
node packages/netabrowser-cli/dist/bin/main.js snapshot --session=test
```
看 url + 内容含 `"origin": "210.51.27.112"`
- [ ] **Step 5: creepjs 反检测 trust score**
打开 https://abrahamjuliot.github.io/creepjs/,肉眼记 trust score。预期 ≥ 70%。
- [ ] **Step 6: 拟人化档位延迟测量**
```bash
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=full
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=fast
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=off
```
预期full ~2-5s, fast ~0.3-0.7s, off <0.2s
- [ ] **Step 7: HTTP 直接调,验证 auth**
**dev 旁路验证**
```bash
# dev 模式NODE_ENV=developmentNETA_TRAY_SECRET 未设)
curl http://localhost:8003/admin/browser-daemon/list
# 200dev 旁路允许 loopback
# 非 loopback 仍 403
curl --interface 8.8.8.8 http://localhost:8003/admin/browser-daemon/list
# 403
```
**prod 风格验证**
```bash
# 杀掉 backend重新启动 prod 模式
NODE_ENV=production NETA_TRAY_SECRET=test-prod-secret pnpm start
# 无 secret → 401
curl http://localhost:8003/admin/browser-daemon/list
# 401
# 错误 secret → 401
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: wrong"
# 401
# 正确 secret → 200
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: test-prod-secret"
# 200
```
- [ ] **Step 8: 软上限验证**
`maxActiveSessions=2`,启动 3 个 session 看第 3 个返回 503。
- [ ] **Step 9: backend 重启 ref 失效**
ctrl+C 关 backend重启 `pnpm dev`,调 `click e1 --session=test`
预期报错session 不存在或 ref 失效),需要先 `open``snapshot``click`
- [ ] **Step 10: 安装包冒烟**
把 setup.exe 装到全新 Windows启动托盘 → 后端启动 → cli 跑 step 4-5。
- [ ] **Step 11: Checkpoint**
`Task 22 完成`:所有验收标准通过。
---
## Task 23文档更新 + 路线图
**Files:**
- Modify: `docs/superpowers/specs/2026-05-04-netabrowser-cli-s1-design.md`(变更日志)
- Modify: `docs/superpowers/specs/2026-05-03-geo-master-roadmap.md`(如存在则更新)
- [ ] **Step 1: 更新 spec §13 变更日志**
加一行:"2026-XX-XX 实施完成,所有验收标准通过,准备进入 S2"。
- [ ] **Step 2: 更新 geo 路线图**
如有 geo master-roadmap标注 netabrowser-cli S1 完成 → 解锁 geo BrowserAutomationService 迁移。
- [ ] **Step 3: 提示 user 走完整 git commit**
```bash
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
git status
# 用户审核后统一 commit如其要求
```
- [ ] **Step 4: Checkpoint**
`Task 23 完成`:文档同步。等待 user 触发统一 commit。
---
## 实施期间约束
- ❌ 不写 SQL 文件(如果有数据库需求,用 mcp mysql 工具直接 INSERT
- ❌ 每个 task **不单独 git commit**(用户要求全部完成 + 联调通过后统一提交)
- ❌ 不修改现有 playwright-cli / patchwright-cli skill
- ❌ 不在 controller 中加业务逻辑(只能是 thin wrapper§5.3.1 强制)
- ❌ HTTP API 必须强制 loopback + secret不能例外
- ❌ 不实现平台特定补丁(小红书/抖音/淘宝等)→ S2
---
## 实施完成后
- 调用 `superpowers:verification-before-completion` 逐条核对验收标准
- 调用 `superpowers:requesting-code-review` 触发代码审查
- 用户测试通过后统一 git commit
- 更新 spec §13 变更日志
- 进入下一个子项目geo BrowserAutomationService 迁移到 BrowserDaemonServiceS2