# netabrowser-cli S1 Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. > > **上位文档**:[`../specs/2026-05-04-netabrowser-cli-s1-design.md`](../specs/2026-05-04-netabrowser-cli-s1-design.md) > **Git 策略**:用户要求"全部完成 + 联调通过后统一提交"。每 task 末尾标注 `Checkpoint`(仅作进度标记,不做 git commit)。 **Goal:** 在 Neta monorepo 内交付 netabrowser-cli S1 基础设施:BrowserDaemonService(嵌入 backend)+ netabrowser-cli 二进制 + skill 元数据,提供反风控+拟人化的浏览器自动化能力,统一服务于 NetaClaw Agent 探索期(CLI)和后端业务固化期(@Inject service)。 **Architecture:** Service-First 架构:核心是 backend 内的 `BrowserDaemonService`(Midway @Singleton)持有 `Map`;CLI 是 thin client 通过 HTTP loopback + secret 调 backend;底层用 `patchright` (npm) 启动 vendored `neta-chromium` (fingerprint-chromium);拟人化用 `ghost-cursor` + 自加套,分 full/fast/off 三档;session-name 串行化锁;LRU 软上限调度。 **Tech Stack:** Midway.js 3.20 / TypeScript 5.9 / `patchright` (npm) / `ghost-cursor` (npm) / fingerprint-chromium 144.0.7559.132 (vendored 二进制) / commander (CLI 解析) / jest+ts-jest / Inno Setup / yao-pkg --- ## ⚠️ v2 修复总览(架构 review 后必读) 本 plan 经过架构师交叉验证发现 8 项 P0/P1 问题,已就地修复。实施时优先理解这些修复点: | # | 问题 | 修复落点 | |---|---|---| | **P0-1** | Controller 路径与 Cool Admin 风格冲突 | Task 11 全部 controller 改用 `@Provide()` + `@Controller('/admin/browser-daemon/')` + 方法相对路径,参考 `netaclaw/controller/admin/agent_channel.ts` 风格 | | **P0-2** | dev 模式 dataDir = `/dist`,build 清空状态 | 新增 Task 4b:`resolveBrowserDataDir()` 独立解析(dev 模式 → `/.netabrowser-data/`,prod → `/.netabrowser/`) | | **P0-3** | browser-daemon 模块挂载错位 | Task 10 Step 5 明确:把 `BrowserControlAuthMiddleware` 加到 `netaclaw/config.ts` 的 `middlewares` 数组,依靠 `match()` 仅作用于 `/admin/browser-daemon/*` 路径 | | **P0-4** | ghost-cursor 类型与 patchright 不兼容 | Task 8 Step 3 humanizer 内部 `createCursor(page as any)` cast | | **P1-5** | dev 模式 NETA_TRAY_SECRET 未设 → 全 401 | Task 10 middleware 加 dev 旁路:`process.env.NODE_ENV !== 'production' && expected === ''` 时跳过 secret 校验 | | **P1-6** | CLI runtime-info 路径错位(cwd 不对) | Task 14 候选改为 `/packages/backend/dist/runtime-info.json` + 强制 `NETA_RUNTIME_INFO` 环境变量优先 | | **P1-7** | @Init/@Destroy lifecycle + 60min idle 自动回收缺失 | 新增 Task 9b:`runtime/cleanup.ts` + daemon.service `@Init`/`@Destroy` + scheduler idle timer | | **P1-8** | CLI 与会话级 mode 协同 bug | service 移除 modeMap(service-stateless),CLI 端用环境变量 `NETA_BROWSER_HUMANIZE_MODE` 作"会话级默认",命令级 `--mode` 覆盖 | **额外路径修正**:Task 4 `chromium-launcher` `__dirname` 上溯 **7 层**(不是 6 层)—— 实际路径 `dist/modules/netaclaw/browser-daemon/runtime/` 到 monorepo root 是 7 个 `..`。 **版本锁定**:Task 1 安装命令改为 `pnpm add patchright@1.59.4 ghost-cursor@1.4.2`(已验证兼容版本)。 --- ## 文件结构 ### 后端模块(新增 14 个文件) | 路径 | 职责 | |---|---| | `packages/backend/src/modules/netaclaw/browser-daemon/config.ts` | 模块配置 + `maxActiveSessions` 等 | | `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` | ★ BrowserDaemonService 核心,统一锁 + 调度入口 | | `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts` | 拟人化包装(ghost-cursor + 三档) | | `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts` | neta-chromium 指纹参数生成 | | `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts` | AI ref 协议(spike #1 选定方案) | | `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts` | 路径解析 + patchright launch 包装 | | `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts` | Map + 串行化锁 | | `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` | LRU 软上限调度 | | `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts` | @Destroy 时优雅关闭 | | `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts` | loopback + secret auth | | `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts` | /open /close /list /stats | | `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts` | /click /fill /type /scroll /hover /press | | `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts` | /goto /back /forward /reload | | `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts` | /cookie-list /cookie-set /state-save /state-load | | `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts` | /snapshot /screenshot /eval /run-code | ### netabrowser-cli 包(新增 13 个文件) | 路径 | 职责 | |---|---| | `packages/netabrowser-cli/package.json` | bin 声明 + patchright/commander 等依赖 | | `packages/netabrowser-cli/tsconfig.json` | TS 配置 | | `packages/netabrowser-cli/src/bin/main.ts` | CLI 入口(commander 解析) | | `packages/netabrowser-cli/src/client/runtime-info.ts` | 读 backend runtime-info.json | | `packages/netabrowser-cli/src/client/http-client.ts` | HTTP 请求 + secret header | | `packages/netabrowser-cli/src/output/formatter.ts` | --raw vs 默认 + 错误格式化 | | `packages/netabrowser-cli/src/commands/session.ts` | open/close/list(共用一个文件) | | `packages/netabrowser-cli/src/commands/interaction.ts` | click/fill/type/scroll/hover/press | | `packages/netabrowser-cli/src/commands/navigation.ts` | goto/back/forward/reload | | `packages/netabrowser-cli/src/commands/state.ts` | cookie / state | | `packages/netabrowser-cli/src/commands/inspect.ts` | snapshot/screenshot/eval/run-code | ### 测试文件(新增 11 个) | 路径 | 测试对象 | |---|---| | `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts` | 路径解析 + launch args 拼装 | | `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts` | seed → args 派生 | | `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts` | 三档行为 + 注入 random source 验证延迟范围 | | `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts` | 串行化锁竞态 | | `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts` | LRU + 软上限 + 队列 | | `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts` | ref 协议契约 | | `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts` | 编排:open/click/cookie/close mock 全链路 | | `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts` | loopback + secret 校验 | | `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts` | HTTP ↔ service 等价性 contract | | `packages/netabrowser-cli/tests/cli-parse.test.ts` | commander 参数解析 | | `packages/netabrowser-cli/tests/output-formatter.test.ts` | 输出格式 | ### Skill 元数据(新增 5 个文件) | 路径 | 职责 | |---|---| | `packages/backend/skills/netabrowser-cli/SKILL.md` | 给 NetaClaw Agent 看的命令清单 | | `packages/backend/skills/netabrowser-cli/references/humanization.md` | 三档详解 | | `packages/backend/skills/netabrowser-cli/references/fingerprint.md` | 指纹参数 | | `packages/backend/skills/netabrowser-cli/references/proxy.md` | 代理配置 | | `packages/backend/skills/netabrowser-cli/references/examples.md` | 典型场景 | ### 集成改动 | 路径 | 改动 | |---|---| | `pnpm-workspace.yaml` | 新增 `packages/netabrowser-cli` | | `packages/backend/package.json` | 新增 `patchright` + `ghost-cursor` 依赖 | | `packages/backend/installer/setup.iss` | `[Files]` 段加 chromium/win64 + netabrowser-cli.exe | | `packages/backend/scripts/build-windows-installer.js` | 复制 chromium 二进制步骤 | | `packages/backend/scripts/pkg-build.js` | 增加 netabrowser-cli pkg 编译 | --- ## 实施依赖图 ``` Phase 0: T0(包脚手架) → T1(npm 依赖) ↓ Phase 1: T2(Spike #1 ref 协议) ‖ T3(Spike #2 ghost-cursor 兼容) ← 阻塞门 ↓ Phase 2: T4(launcher) → T5(registry+lock) → T6(scheduler) → T7(open/close) ↓ T8(humanizer + click/fill) → T9(state/cookie/snapshot) ↓ Phase 3: T10(auth middleware) → T11(controllers) → T12(contract test) ↓ Phase 4: T13(cli bin) → T14(client) → T15-17(命令实现) → T18(formatter) ↓ Phase 5: T19-20(skill 元数据) ↓ Phase 6: T21(pkg build) → T22(installer) → T23(路径解析) ↓ Phase 7: T24(联调冒烟) → T25(性能验证) → T26(文档) ``` --- ## Task 0:netabrowser-cli 包脚手架 **Files:** - Create: `packages/netabrowser-cli/package.json` - Create: `packages/netabrowser-cli/tsconfig.json` - Modify: `pnpm-workspace.yaml` - [ ] **Step 1: 把 netabrowser-cli 加入 pnpm workspace** 读 `pnpm-workspace.yaml`,确认含有 `packages/*` 通配符(应该已有)。如无 `packages/*` 通配符,加上: ```yaml packages: - 'packages/*' ``` 无需修改的话跳过。 - [ ] **Step 2: 创建 netabrowser-cli/package.json** ```json { "name": "@neta/netabrowser-cli", "version": "0.1.0", "description": "Neta 反风控+拟人化浏览器自动化 CLI", "private": true, "type": "module", "bin": { "netabrowser-cli": "./dist/bin/main.js" }, "scripts": { "build": "tsc -p tsconfig.json", "test": "jest" }, "dependencies": { "commander": "^12.0.0", "axios": "^1.7.0" }, "devDependencies": { "@types/node": "^22.0.0", "jest": "^29.7.0", "ts-jest": "^29.1.5", "typescript": "^5.9.0" } } ``` - [ ] **Step 3: 创建 netabrowser-cli/tsconfig.json** ```json { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "outDir": "dist", "rootDir": "src", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "resolveJsonModule": true, "declaration": false }, "include": ["src/**/*"] } ``` - [ ] **Step 4: 验证 pnpm 识别 workspace** ```bash cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo pnpm install pnpm list -r --depth=-1 | grep netabrowser-cli ``` 预期:列表中出现 `@neta/netabrowser-cli` - [ ] **Step 5: Checkpoint** `Task 0 完成`:包注册到 workspace,可以独立 build/test。 --- ## Task 1:backend 安装 patchright + ghost-cursor 依赖 **Files:** - Modify: `packages/backend/package.json` - [ ] **Step 1: 在 backend 安装 patchright(npm 包,锁定版本)** ```bash cd packages/backend pnpm add patchright@1.59.4 ``` > 锁定 1.59.4:经 review 验证此版本兼容 Node 22 + Midway 3.20 + neta-chromium 144.0.7559.132。后续升级前需重测兼容性。 - [ ] **Step 2: 安装 ghost-cursor(锁定版本)** ```bash pnpm add ghost-cursor@1.4.2 ``` - [ ] **Step 3: 验证 patchright 能 import** 跑一段临时脚本: ```bash cd packages/backend node -e "import('patchright').then(p => console.log('patchright loaded:', Object.keys(p).join(',')))" ``` 预期输出含 `chromium,firefox,webkit,...` - [ ] **Step 4: 验证 ghost-cursor 能 import** ```bash node -e "import('ghost-cursor').then(p => console.log('ghost-cursor loaded:', Object.keys(p).join(',')))" ``` 预期含 `createCursor,path,...` - [ ] **Step 5: Checkpoint** `Task 1 完成`:依赖装好,可以在 service 代码 import 使用。 --- ## Task 2 [SPIKE #1]:AI Ref 协议方案验证 > 这是 spec §11 的阻塞门 spike。**必须先做,否则后续 interaction 命令无法实现**。 **目标**:选定一种 AI ref 协议方案并 demo 跑通。 候选方案: - **A. 自实现**:snapshot 时给 DOM 注入 `data-ai-ref="e15"` 属性 + 内存维护 `Map>` - **B. 改用 selector**:命令直接接收 selector 字符串(`click 'button:has-text("登录")'`),不要 ref - **C. vendor playwright-cli ref 代码**:剥离 playwright-cli 内部 ref 实现引入 netabrowser-cli **Files:** - Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts` - [ ] **Step 1: 写 spike 测试(方案 A,先验证最简实现)** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts import { chromium } from 'patchright'; import * as path from 'node:path'; import * as fs from 'node:fs'; const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe'); describe('Spike #1: AI ref 协议(方案 A 自实现)', () => { jest.setTimeout(60000); it('A: snapshot 注入 data-ai-ref,可基于 ref 定位元素', async () => { if (!fs.existsSync(CHROME)) { console.warn('SKIP: neta-chromium not found at', CHROME); return; } const tmpProfile = path.resolve(process.cwd(), '.spike-profile-1'); const ctx = await chromium.launchPersistentContext(tmpProfile, { executablePath: CHROME, headless: true, args: ['--fingerprint=12345'], }); const page = ctx.pages()[0] || (await ctx.newPage()); await page.goto('data:text/html,Link'); // 1. snapshot:给所有可交互元素注入 data-ai-ref const refs = await page.evaluate(() => { const interactive = Array.from(document.querySelectorAll('button, a, input, select, textarea')); const out: { ref: string; tag: string; text: string }[] = []; interactive.forEach((el, i) => { const ref = `e${i + 1}`; el.setAttribute('data-ai-ref', ref); out.push({ ref, tag: el.tagName.toLowerCase(), text: (el.textContent || '').trim() }); }); return out; }); expect(refs).toEqual([ { ref: 'e1', tag: 'button', text: 'Login' }, { ref: 'e2', tag: 'a', text: 'Link' }, ]); // 2. 通过 ref click await page.locator('[data-ai-ref="e1"]').click(); // 不报错 = 通过 await ctx.close(); fs.rmSync(tmpProfile, { recursive: true, force: true }); }); }); ``` - [ ] **Step 2: 跑 spike 测试** ```bash cd packages/backend pnpm jest test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts -t 'A:' ``` 预期:PASS(含 fingerprint chromium 已在 vendor 里) - [ ] **Step 3: 评审 spike 结果** 如果方案 A PASS,写入决策文件: ```bash cat > packages/backend/src/modules/netaclaw/browser-daemon/SPIKE_DECISION.md << 'EOF' # Spike #1 决策:AI Ref 协议 **选定方案 A:自实现 data-ai-ref 注入** ## 理由 - 实现简单(一段 page.evaluate JS 即可) - 不引入额外依赖 - 完全可控 ## 实现要点 1. snapshot 命令时调用 `injectRefs(page)`,给所有可交互元素加 `data-ai-ref="eN"` 2. service 内存维护 `Map>` 用于审计/调试 3. interaction 命令收到 ref 后用 `page.locator('[data-ai-ref="${ref}"]')` 定位 4. 页面跳转/重新 snapshot 后旧 ref 失效(DOM 变化) ## 不选 B/C 的理由 - B(selector):AI 写 selector 容易错;ref 抽象更友好 - C(vendor playwright-cli):playwright-cli 用 socket+state 同步,移植成本高 EOF ``` 如果方案 A FAIL(极不可能),降级 B/C 重写 spike。 - [ ] **Step 4: Checkpoint** `Task 2 完成`:方案 A 通过,决策记录在 `SPIKE_DECISION.md`。后续 snapshot-ref.service.ts 实现该方案。 --- ## Task 3 [SPIKE #2]:ghost-cursor on patchright 兼容性验证 **Files:** - Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts` - [ ] **Step 1: 写 spike 测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts import { chromium } from 'patchright'; import { createCursor } from 'ghost-cursor'; import * as path from 'node:path'; import * as fs from 'node:fs'; const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe'); describe('Spike #2: ghost-cursor on patchright + neta-chromium', () => { jest.setTimeout(60000); it('cursor.click 能在 patchright 启动的 neta-chromium 上工作', async () => { if (!fs.existsSync(CHROME)) { console.warn('SKIP: neta-chromium not found'); return; } const tmpProfile = path.resolve(process.cwd(), '.spike-profile-2'); const ctx = await chromium.launchPersistentContext(tmpProfile, { executablePath: CHROME, headless: true, args: ['--fingerprint=99999'], }); const page = ctx.pages()[0] || (await ctx.newPage()); await page.goto('data:text/html,'); // 创建 ghost cursor const cursor = createCursor(page as any); // 拿目标坐标 const box = await page.locator('#b').boundingBox(); expect(box).not.toBeNull(); // 用 ghost-cursor 移动并点击(贝塞尔轨迹) await cursor.moveTo({ x: box!.x + box!.width / 2, y: box!.y + box!.height / 2 }); await page.mouse.down(); await page.waitForTimeout(80); await page.mouse.up(); // 等待 click 生效 await page.waitForFunction(() => document.title === 'CLICKED', null, { timeout: 5000 }); expect(await page.title()).toBe('CLICKED'); // 验证 navigator.webdriver 仍为 false(patchright 反检测有效) const wd = await page.evaluate(() => navigator.webdriver); expect(wd).toBe(false); await ctx.close(); fs.rmSync(tmpProfile, { recursive: true, force: true }); }); }); ``` - [ ] **Step 2: 跑 spike** ```bash pnpm jest test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts ``` 预期:PASS(含 navigator.webdriver === false) - [ ] **Step 3: 写决策** 如果通过,append `SPIKE_DECISION.md`: ```markdown # Spike #2 决策:ghost-cursor 兼容 ✅ ghost-cursor 在 patchright + neta-chromium 上工作正常。 - cursor.moveTo 走贝塞尔轨迹,patchright 没拦截 mouse 事件 - click 后 navigator.webdriver 仍 false(patchright 反检测维持) humanizer.service.ts 直接 `import { createCursor } from 'ghost-cursor'` 即可。 ``` 如果失败:选用纯自写贝塞尔(替代方案,备选脚本另准备)。 - [ ] **Step 4: Checkpoint** `Task 3 完成`:ghost-cursor 兼容性确认,可投入 humanizer 实现。 --- ## Task 4:chromium-launcher(路径解析 + launch args 拼装) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts` - [ ] **Step 1: 写 fingerprint 测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts import { FingerprintService } from '../../../../src/modules/netaclaw/browser-daemon/service/fingerprint.service.js'; describe('FingerprintService', () => { const svc = new FingerprintService(); it('fromSeed: 仅 seed 生成完整 args', () => { const args = svc.fromSeed(12345); expect(args).toEqual(expect.arrayContaining([ '--fingerprint=12345', '--fingerprint-platform=Windows', '--fingerprint-platform-version=10.0.0', '--fingerprint-brand=Chrome', '--fingerprint-hardware-concurrency=8', '--fingerprint-language=zh-CN', ])); }); it('merge: 指定字段覆盖默认', () => { const args = svc.merge({ seed: 1, language: 'en-US', hardwareConcurrency: 16 }); expect(args).toContain('--fingerprint=1'); expect(args).toContain('--fingerprint-language=en-US'); expect(args).toContain('--fingerprint-hardware-concurrency=16'); }); it('seed 不同生成不同 fingerprint arg', () => { expect(svc.fromSeed(1)).toContain('--fingerprint=1'); expect(svc.fromSeed(2)).toContain('--fingerprint=2'); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash cd packages/backend && pnpm jest fingerprint.service ``` 预期:FAIL(service 不存在) - [ ] **Step 3: 实现 FingerprintService** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts import { Provide, Scope, ScopeEnum } from '@midwayjs/core'; export interface FingerprintParams { seed?: number; platform?: 'Windows' | 'macOS' | 'Linux'; platformVersion?: string; brand?: string; brandVersion?: string; hardwareConcurrency?: number; language?: string; timezone?: string; } @Provide() @Scope(ScopeEnum.Singleton) export class FingerprintService { /** 仅 seed → 全套默认 args */ fromSeed(seed: number): string[] { return this.merge({ seed }); } /** 自定义参数 + 默认值 → args */ merge(p: FingerprintParams): string[] { const args: string[] = []; if (p.seed != null) args.push(`--fingerprint=${p.seed}`); args.push(`--fingerprint-platform=${p.platform ?? 'Windows'}`); args.push(`--fingerprint-platform-version=${p.platformVersion ?? '10.0.0'}`); args.push(`--fingerprint-brand=${p.brand ?? 'Chrome'}`); if (p.brandVersion) args.push(`--fingerprint-brand-version=${p.brandVersion}`); args.push(`--fingerprint-hardware-concurrency=${p.hardwareConcurrency ?? 8}`); args.push(`--fingerprint-language=${p.language ?? 'zh-CN'}`); if (p.timezone) args.push(`--fingerprint-timezone=${p.timezone}`); return args; } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest fingerprint.service ``` 预期:PASS(3 tests) - [ ] **Step 5: 写 chromium-launcher 测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts import { resolveChromiumPath, buildLaunchArgs } from '../../../../src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.js'; import * as path from 'node:path'; describe('chromium-launcher', () => { describe('resolveChromiumPath', () => { afterEach(() => { delete process.env.NETA_CHROMIUM_PATH; }); it('优先环境变量', () => { process.env.NETA_CHROMIUM_PATH = '/custom/chrome.exe'; expect(resolveChromiumPath({ isPkg: false })).toBe('/custom/chrome.exe'); }); it('pkg 模式:execPath 同目录 chromium/win64/chrome.exe', () => { const result = resolveChromiumPath({ isPkg: true, execPath: 'C:/Program Files/Neta/backend.exe' }); expect(result).toMatch(/Program Files\/Neta\/chromium\/win64\/chrome\.exe$/); }); it('dev 模式:monorepo packages/netabrowser-cli/chromium/win64/chrome.exe', () => { const result = resolveChromiumPath({ isPkg: false }); expect(result).toMatch(/netabrowser-cli\/chromium\/win64\/chrome\.exe$/); }); }); describe('buildLaunchArgs', () => { it('合并指纹 args + 用户自定义 args', () => { const args = buildLaunchArgs({ fingerprintArgs: ['--fingerprint=42', '--fingerprint-language=zh-CN'], extraArgs: ['--disable-popup-blocking'], }); expect(args).toEqual(expect.arrayContaining([ '--fingerprint=42', '--fingerprint-language=zh-CN', '--disable-popup-blocking', ])); }); }); }); ``` - [ ] **Step 6: 实现 chromium-launcher** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts import * as path from 'node:path'; export interface ResolveOpts { isPkg?: boolean; execPath?: string; } /** 解析 neta-chromium 可执行文件路径 */ export function resolveChromiumPath(opts: ResolveOpts = {}): string { if (process.env.NETA_CHROMIUM_PATH) return process.env.NETA_CHROMIUM_PATH; const isPkg = opts.isPkg ?? !!(process as any).pkg; if (isPkg) { const execPath = opts.execPath ?? process.execPath; return path.join(path.dirname(execPath), 'chromium', 'win64', 'chrome.exe').replace(/\\/g, '/'); } // dev 模式:从 backend src 出发,定位 monorepo 内 packages/netabrowser-cli/chromium/win64/chrome.exe // 编译后 __dirname = packages/backend/dist/modules/netaclaw/browser-daemon/runtime // 路径计算:runtime → browser-daemon → netaclaw → modules → dist → backend → packages → root(7 层 ..) // 然后 root/packages/netabrowser-cli/chromium/win64/chrome.exe return path.resolve(__dirname, '../../../../../../../packages/netabrowser-cli/chromium/win64/chrome.exe').replace(/\\/g, '/'); } export interface LaunchArgsOpts { fingerprintArgs: string[]; extraArgs?: string[]; } export function buildLaunchArgs(opts: LaunchArgsOpts): string[] { return [...opts.fingerprintArgs, ...(opts.extraArgs ?? [])]; } ``` - [ ] **Step 7: 跑 chromium-launcher 测试** ```bash pnpm jest chromium-launcher ``` 预期:PASS(4 tests) - [ ] **Step 8: Checkpoint** `Task 4 完成`:路径解析 + 指纹参数生成 + launch args 拼装可用。 --- ## Task 4b:BrowserDataDir 独立解析(**新增 P0 修复**) **问题**:`comm/data-dir.ts:resolveDataDir()` 在 dev 模式 fallback 到 `/dist`。把 `browser-profiles/` `states/` 放 dist 下,每次 `pnpm build` 会清空,状态全丢。 **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts` - [ ] **Step 1: 写测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts import { resolveBrowserDataDir, getProfileDir, getStateDir } from '../../../../src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.js'; import * as path from 'node:path'; describe('BrowserDataDir', () => { afterEach(() => { delete process.env.NETA_BROWSER_DATA_DIR; delete process.env.NETA_DATA_DIR; }); it('优先 NETA_BROWSER_DATA_DIR 环境变量', () => { process.env.NETA_BROWSER_DATA_DIR = '/custom/browser'; expect(resolveBrowserDataDir({ isPkg: false })).toBe('/custom/browser'); }); it('pkg 模式:dataDir/.netabrowser', () => { const r = resolveBrowserDataDir({ isPkg: true, execDir: 'C:/Program Files/Neta' }); expect(r.replace(/\\/g, '/')).toBe('C:/Program Files/Neta/data/.netabrowser'); }); it('dev 模式:monorepo/.netabrowser-data(不复用 dist)', () => { const r = resolveBrowserDataDir({ isPkg: false, monorepoRoot: '/repo' }); expect(r.replace(/\\/g, '/')).toBe('/repo/.netabrowser-data'); }); it('getProfileDir / getStateDir 返回子路径', () => { process.env.NETA_BROWSER_DATA_DIR = '/x'; expect(getProfileDir('s1').replace(/\\/g, '/')).toBe('/x/profiles/s1'); expect(getStateDir('s1').replace(/\\/g, '/')).toBe('/x/states/s1.json'); }); }); ``` - [ ] **Step 2: 实现** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts import * as path from 'node:path'; import { resolveDataDir } from '../../../../comm/data-dir.js'; export interface ResolveBrowserDataDirOpts { isPkg?: boolean; execDir?: string; monorepoRoot?: string; } /** * netabrowser-cli 自己的状态根目录,独立于 backend dist/。 * - prod: /.netabrowser/ * - dev: /.netabrowser-data/ (不放 dist 下,防 build 清空) */ export function resolveBrowserDataDir(opts: ResolveBrowserDataDirOpts = {}): string { if (process.env.NETA_BROWSER_DATA_DIR) return path.resolve(process.env.NETA_BROWSER_DATA_DIR); const isPkg = opts.isPkg ?? !!(process as any).pkg; if (isPkg) { const dataDir = resolveDataDir({ isPkg: true, execDir: opts.execDir }); return path.join(dataDir, '.netabrowser'); } // dev: 从 backend 出发上溯到 monorepo root const monorepoRoot = opts.monorepoRoot ?? path.resolve(__dirname, '../../../../../../../'); return path.join(monorepoRoot, '.netabrowser-data'); } export function getProfileDir(sessionName: string): string { return path.join(resolveBrowserDataDir(), 'profiles', sessionName); } export function getStateDir(sessionName: string): string { return path.join(resolveBrowserDataDir(), 'states', `${sessionName}.json`); } ``` - [ ] **Step 3: 跑测试看通过** ```bash pnpm jest browser-data-dir ``` 预期:PASS(4 tests) - [ ] **Step 4: 修改 Task 7 daemon.service 中的 import** 将 daemon.service.ts 的: ```typescript import { resolveDataDir } from '../../../../comm/data-dir.js'; // ... const profileRoot = path.join(resolveDataDir(), 'browser-profiles'); const profilePath = path.join(profileRoot, opts.profileDir ?? opts.sessionName); ``` 改为: ```typescript import { getProfileDir } from '../runtime/browser-data-dir.js'; // ... const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName); ``` 同样将 saveState/loadState 用到的状态文件路径改为 `getStateDir(sessionName)`。 - [ ] **Step 5: 更新 .gitignore** 把 `.netabrowser-data/` 加入 monorepo 根 `.gitignore`: ```bash echo ".netabrowser-data/" >> C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo/.gitignore ``` - [ ] **Step 6: Checkpoint** `Task 4b 完成`:BrowserDataDir 独立,不复用 dist,build 不会清空状态。 --- ## Task 5:SessionRegistry(含串行化锁) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts` - [ ] **Step 1: 写测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts import { SessionRegistry } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js'; describe('SessionRegistry', () => { let registry: SessionRegistry; beforeEach(() => { registry = new SessionRegistry(); }); it('register/get/has/list 基础 CRUD', () => { const fakeCtx = { id: 'A' } as any; registry.register('s1', fakeCtx); expect(registry.has('s1')).toBe(true); expect(registry.get('s1')).toBe(fakeCtx); expect(registry.list()).toEqual([{ sessionName: 's1', context: fakeCtx, lastUsedAt: expect.any(Date) }]); registry.unregister('s1'); expect(registry.has('s1')).toBe(false); }); it('touch 更新 lastUsedAt', async () => { registry.register('s1', { id: 'A' } as any); const before = registry.list()[0].lastUsedAt; await new Promise(r => setTimeout(r, 10)); registry.touch('s1'); const after = registry.list()[0].lastUsedAt; expect(after.getTime()).toBeGreaterThan(before.getTime()); }); it('withLock: 同 sessionName 串行化', async () => { const order: string[] = []; const job = (id: string, ms: number) => async () => { order.push(`start-${id}`); await new Promise(r => setTimeout(r, ms)); order.push(`end-${id}`); }; await Promise.all([ registry.withLock('s1', job('A', 30)), registry.withLock('s1', job('B', 5)), registry.withLock('s1', job('C', 5)), ]); expect(order).toEqual(['start-A', 'end-A', 'start-B', 'end-B', 'start-C', 'end-C']); }); it('withLock: 不同 sessionName 并发', async () => { const order: string[] = []; await Promise.all([ registry.withLock('s1', async () => { order.push('s1-start'); await new Promise(r => setTimeout(r, 30)); order.push('s1-end'); }), registry.withLock('s2', async () => { order.push('s2-start'); await new Promise(r => setTimeout(r, 10)); order.push('s2-end'); }), ]); // s2 应该在 s1 完成前先 end expect(order.indexOf('s2-end')).toBeLessThan(order.indexOf('s1-end')); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash pnpm jest session-registry ``` 预期:FAIL - [ ] **Step 3: 实现** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts import { Provide, Scope, ScopeEnum } from '@midwayjs/core'; export interface SessionEntry { sessionName: string; context: any; // patchright BrowserContext lastUsedAt: Date; } @Provide() @Scope(ScopeEnum.Singleton) export class SessionRegistry { private entries = new Map(); private locks = new Map>(); register(sessionName: string, context: any): void { this.entries.set(sessionName, { sessionName, context, lastUsedAt: new Date() }); } unregister(sessionName: string): void { this.entries.delete(sessionName); } has(sessionName: string): boolean { return this.entries.has(sessionName); } get(sessionName: string): any | undefined { return this.entries.get(sessionName)?.context; } touch(sessionName: string): void { const e = this.entries.get(sessionName); if (e) e.lastUsedAt = new Date(); } list(): SessionEntry[] { return [...this.entries.values()]; } /** 同 sessionName 操作严格串行化 */ async withLock(sessionName: string, fn: () => Promise): Promise { const prev = this.locks.get(sessionName) ?? Promise.resolve(); let release!: () => void; const next = new Promise(r => (release = r)); this.locks.set(sessionName, prev.then(() => next)); try { await prev; return await fn(); } finally { release(); if (this.locks.get(sessionName) === next) this.locks.delete(sessionName); } } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest session-registry ``` 预期:PASS(4 tests) - [ ] **Step 5: Checkpoint** `Task 5 完成`:SessionRegistry 含锁机制,并发安全。 --- ## Task 6:SessionScheduler(LRU 软上限 + 队列) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts` - [ ] **Step 1: 写测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts import { SessionScheduler } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js'; describe('SessionScheduler', () => { it('未触达上限直接通过', async () => { const sch = new SessionScheduler({ maxActiveSessions: 5 }); sch.recordActivate('s1'); const slot = await sch.acquireSlot('s2', 'normal', false); expect(slot.granted).toBe(true); expect(slot.evictedSessionName).toBeUndefined(); }); it('触达上限:LRU 回收最久未用的 idle session', async () => { const sch = new SessionScheduler({ maxActiveSessions: 2, idleTimeoutMs: 1000 }); sch.recordActivate('s1'); await new Promise(r => setTimeout(r, 5)); sch.recordActivate('s2'); sch.markIdle('s1'); sch.markIdle('s2'); // s1 比 s2 先 idle,所以 s1 是 LRU const slot = await sch.acquireSlot('s3', 'normal', false); expect(slot.granted).toBe(true); expect(slot.evictedSessionName).toBe('s1'); }); it('全部 active 无 idle 可回收:fail-fast 503', async () => { const sch = new SessionScheduler({ maxActiveSessions: 1 }); sch.recordActivate('s1'); const slot = await sch.acquireSlot('s2', 'normal', false); expect(slot.granted).toBe(false); expect(slot.reason).toBe('NO_IDLE_SESSION_TO_EVICT'); }); it('high 优先级抢占 low 优先级', async () => { const sch = new SessionScheduler({ maxActiveSessions: 1 }); sch.recordActivate('s1', 'low'); const slot = await sch.acquireSlot('s2', 'high', false); expect(slot.granted).toBe(true); expect(slot.evictedSessionName).toBe('s1'); }); it('queue=true:等待空位(最长 30s)', async () => { const sch = new SessionScheduler({ maxActiveSessions: 1, queueTimeoutMs: 200 }); sch.recordActivate('s1'); const p = sch.acquireSlot('s2', 'normal', true); setTimeout(() => sch.markIdle('s1'), 50); setTimeout(() => sch.recordDeactivate('s1'), 60); const slot = await p; expect(slot.granted).toBe(true); }); it('getStats 返回 active/idle/total', () => { const sch = new SessionScheduler({ maxActiveSessions: 5 }); sch.recordActivate('s1'); sch.recordActivate('s2'); sch.markIdle('s1'); expect(sch.getStats()).toEqual({ activeCount: 1, idleCount: 1, totalCount: 2, maxActiveSessions: 5, }); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash pnpm jest session-scheduler ``` - [ ] **Step 3: 实现** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts import { Provide, Scope, ScopeEnum } from '@midwayjs/core'; export type Priority = 'low' | 'normal' | 'high'; const PRIORITY_RANK: Record = { low: 0, normal: 1, high: 2 }; interface SessionState { sessionName: string; priority: Priority; status: 'active' | 'idle'; activatedAt: Date; lastUsedAt: Date; } export interface SchedulerOptions { maxActiveSessions?: number; idleTimeoutMs?: number; queueTimeoutMs?: number; } export interface AcquireResult { granted: boolean; reason?: string; evictedSessionName?: string; } @Provide() @Scope(ScopeEnum.Singleton) export class SessionScheduler { private sessions = new Map(); private waiters: Array<{ resolve: (v: AcquireResult) => void; sessionName: string; priority: Priority; expireAt: number }> = []; private maxActiveSessions: number; private queueTimeoutMs: number; constructor(opts: SchedulerOptions = {}) { this.maxActiveSessions = opts.maxActiveSessions ?? 50; this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000; } recordActivate(sessionName: string, priority: Priority = 'normal'): void { const now = new Date(); this.sessions.set(sessionName, { sessionName, priority, status: 'active', activatedAt: now, lastUsedAt: now }); this.tryProcessWaiters(); } markIdle(sessionName: string): void { const s = this.sessions.get(sessionName); if (s) s.status = 'idle'; this.tryProcessWaiters(); } recordDeactivate(sessionName: string): void { this.sessions.delete(sessionName); this.tryProcessWaiters(); } touch(sessionName: string): void { const s = this.sessions.get(sessionName); if (s) { s.status = 'active'; s.lastUsedAt = new Date(); } } async acquireSlot(sessionName: string, priority: Priority, queue: boolean): Promise { const direct = this.tryAcquireImmediate(sessionName, priority); if (direct.granted || !queue) return direct; if (direct.reason !== 'NO_IDLE_SESSION_TO_EVICT' && direct.reason !== 'AT_CAPACITY') { return direct; } // 排队 return new Promise(resolve => { const expireAt = Date.now() + this.queueTimeoutMs; const waiter = { resolve, sessionName, priority, expireAt }; this.waiters.push(waiter); const timer = setTimeout(() => { const i = this.waiters.indexOf(waiter); if (i >= 0) { this.waiters.splice(i, 1); resolve({ granted: false, reason: 'QUEUE_TIMEOUT' }); } }, this.queueTimeoutMs); // resolve 时清除 timer const origResolve = waiter.resolve; waiter.resolve = (v) => { clearTimeout(timer); origResolve(v); }; }); } private tryAcquireImmediate(sessionName: string, priority: Priority): AcquireResult { if (this.sessions.has(sessionName)) { // 已存在:activate 并返回成功 this.touch(sessionName); return { granted: true }; } const activeCount = [...this.sessions.values()].filter(s => s.status === 'active').length; if (activeCount < this.maxActiveSessions) { return { granted: true }; } // 触达上限:尝试 LRU 回收 idle const idleSessions = [...this.sessions.values()] .filter(s => s.status === 'idle') .sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime()); if (idleSessions.length > 0) { const evict = idleSessions[0]; this.sessions.delete(evict.sessionName); return { granted: true, evictedSessionName: evict.sessionName }; } // 无 idle,尝试优先级抢占 const lowerPriority = [...this.sessions.values()] .filter(s => PRIORITY_RANK[s.priority] < PRIORITY_RANK[priority]) .sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime()); if (lowerPriority.length > 0) { const evict = lowerPriority[0]; this.sessions.delete(evict.sessionName); return { granted: true, evictedSessionName: evict.sessionName }; } return { granted: false, reason: 'NO_IDLE_SESSION_TO_EVICT' }; } private tryProcessWaiters(): void { while (this.waiters.length > 0) { const w = this.waiters[0]; if (Date.now() > w.expireAt) { this.waiters.shift(); w.resolve({ granted: false, reason: 'QUEUE_TIMEOUT' }); continue; } const r = this.tryAcquireImmediate(w.sessionName, w.priority); if (r.granted) { this.waiters.shift(); w.resolve(r); } else { break; } } } getStats(): { activeCount: number; idleCount: number; totalCount: number; maxActiveSessions: number } { const all = [...this.sessions.values()]; return { activeCount: all.filter(s => s.status === 'active').length, idleCount: all.filter(s => s.status === 'idle').length, totalCount: all.length, maxActiveSessions: this.maxActiveSessions, }; } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest session-scheduler ``` 预期:PASS(6 tests) - [ ] **Step 5: Checkpoint** `Task 6 完成`:调度器 LRU+优先级+队列+stats 完整。 --- ## Task 7:BrowserDaemonService.open/close 基础生命周期 **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts` - [ ] **Step 1: 写测试(mock patchright + dependencies)** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts import { BrowserDaemonService } from '../../../../src/modules/netaclaw/browser-daemon/service/daemon.service.js'; describe('BrowserDaemonService.open/close', () => { let svc: BrowserDaemonService; let mockChromium: any; let mockContext: any; let mockPage: any; beforeEach(() => { mockPage = { goto: jest.fn(), close: jest.fn(), url: () => 'about:blank' }; mockContext = { pages: () => [mockPage], newPage: jest.fn().mockResolvedValue(mockPage), close: jest.fn().mockResolvedValue(undefined), storageState: jest.fn().mockResolvedValue({ cookies: [], origins: [] }), }; mockChromium = { launchPersistentContext: jest.fn().mockResolvedValue(mockContext) }; svc = new BrowserDaemonService(); (svc as any).registry = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js').SessionRegistry)(); (svc as any).scheduler = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js').SessionScheduler)({ maxActiveSessions: 5 }); (svc as any).fingerprint = { fromSeed: (s: number) => [`--fingerprint=${s}`] }; (svc as any).chromium = mockChromium; (svc as any).resolveChromiumPath = () => '/fake/chrome.exe'; (svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() }; }); it('open: 启动 chromium + 注册 session', async () => { const r = await svc.open({ sessionName: 's1', url: 'https://example.com', fingerprintSeed: 42 }); expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith( expect.any(String), expect.objectContaining({ executablePath: '/fake/chrome.exe', args: expect.arrayContaining(['--fingerprint=42']), }) ); expect(r.sessionName).toBe('s1'); expect((svc as any).registry.has('s1')).toBe(true); }); it('open: sessionName 已存在 → 409', async () => { await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }); await expect(svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 })) .rejects.toThrow(/already exists|conflict/i); }); it('close: 关闭 context + 注销 session', async () => { await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }); await svc.close('s1'); expect(mockContext.close).toHaveBeenCalled(); expect((svc as any).registry.has('s1')).toBe(false); }); it('list: 返回所有 session 信息', async () => { await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }); const list = svc.list(); expect(list).toEqual([expect.objectContaining({ sessionName: 's1' })]); }); it('proxy 配置传给 patchright', async () => { await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1, proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' }, }); expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith( expect.any(String), expect.objectContaining({ proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' }, }) ); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash pnpm jest daemon.service ``` - [ ] **Step 3: 实现 daemon.service.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts import { Provide, Scope, ScopeEnum, Inject, Logger, ILogger } from '@midwayjs/core'; import * as path from 'node:path'; import * as fs from 'node:fs'; import { chromium } from 'patchright'; import { SessionRegistry } from '../runtime/session-registry.js'; import { SessionScheduler, Priority } from '../runtime/session-scheduler.js'; import { FingerprintService } from './fingerprint.service.js'; import { resolveChromiumPath, buildLaunchArgs } from '../runtime/chromium-launcher.js'; import { getProfileDir } from '../runtime/browser-data-dir.js'; export type HumanizeMode = 'full' | 'fast' | 'off'; export interface OpenOpts { sessionName: string; url: string; fingerprintSeed?: number; proxy?: { server: string; username?: string; password?: string }; profileDir?: string; headed?: boolean; priority?: Priority; queue?: boolean; } export interface SessionInfo { sessionName: string; url: string; pageCount: number; status: 'active' | 'idle'; lastUsedAt: Date; } @Provide() @Scope(ScopeEnum.Singleton) export class BrowserDaemonService { @Inject() registry: SessionRegistry; @Inject() scheduler: SessionScheduler; @Inject() fingerprint: FingerprintService; @Logger() logger: ILogger; // 注入点便于测试覆盖 protected chromium: any = chromium; protected resolveChromiumPath = resolveChromiumPath; /** * 注:v2 修复 P1-8 后 service 不再维护 modeMap(保持 stateless)。 * humanize-mode 完全由调用方每次传: * - CLI 端:`--mode` 命令级 > `NETA_BROWSER_HUMANIZE_MODE` 环境变量 > 默认 'full' * - HTTP/Service 调用方:每次方法调用通过参数传入 */ async open(opts: OpenOpts): Promise<{ sessionName: string; url: string; pageCount: number }> { return this.registry.withLock(opts.sessionName, async () => { if (this.registry.has(opts.sessionName)) { const err: any = new Error(`Session '${opts.sessionName}' already exists`); err.statusCode = 409; throw err; } const slot = await this.scheduler.acquireSlot( opts.sessionName, opts.priority ?? 'normal', opts.queue ?? false, ); if (!slot.granted) { const err: any = new Error(`No slot available: ${slot.reason}`); err.statusCode = 503; err.retryAfter = 30; throw err; } if (slot.evictedSessionName) { await this.closeInternal(slot.evictedSessionName); } const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName); fs.mkdirSync(profilePath, { recursive: true }); const fingerprintArgs = this.fingerprint.fromSeed(opts.fingerprintSeed ?? Math.floor(Math.random() * 100000)); const args = buildLaunchArgs({ fingerprintArgs }); this.logger.info(`[browser-daemon] launching ${opts.sessionName} → ${opts.url}`); const ctx = await this.chromium.launchPersistentContext(profilePath, { executablePath: this.resolveChromiumPath(), headless: !opts.headed, args, proxy: opts.proxy, }); const page = ctx.pages()[0] ?? (await ctx.newPage()); await page.goto(opts.url); this.registry.register(opts.sessionName, ctx); this.scheduler.recordActivate(opts.sessionName, opts.priority ?? 'normal'); return { sessionName: opts.sessionName, url: page.url(), pageCount: ctx.pages().length }; }); } async close(sessionName: string): Promise { return this.registry.withLock(sessionName, () => this.closeInternal(sessionName)); } private async closeInternal(sessionName: string): Promise { const ctx = this.registry.get(sessionName); if (!ctx) return; try { await ctx.close(); } catch (e: any) { this.logger.warn(`[browser-daemon] close ${sessionName} failed: ${e.message}`); } this.registry.unregister(sessionName); this.scheduler.recordDeactivate(sessionName); } list(): SessionInfo[] { return this.registry.list().map(e => { const ctx = e.context; const page = ctx.pages?.()[0]; return { sessionName: e.sessionName, url: page?.url?.() ?? '', pageCount: ctx.pages?.().length ?? 0, status: 'active', lastUsedAt: e.lastUsedAt, }; }); } getStats() { return this.scheduler.getStats(); } getPageOrFail(sessionName: string) { const ctx = this.registry.get(sessionName); if (!ctx) throw new Error(`Session '${sessionName}' not found`); const page = ctx.pages()[0]; if (!page) throw new Error(`Session '${sessionName}' has no page`); this.registry.touch(sessionName); this.scheduler.touch(sessionName); return { ctx, page }; } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest daemon.service ``` 预期:PASS(5 tests) - [ ] **Step 5: Checkpoint** `Task 7 完成`:daemon.service open/close/list 可用,含锁、调度、指纹、launch。 --- ## Task 8:HumanizerService(三档拟人化) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts` - [ ] **Step 1: 写测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts import { HumanizerService } from '../../../../src/modules/netaclaw/browser-daemon/service/humanizer.service.js'; describe('HumanizerService', () => { let svc: HumanizerService; let mockPage: any; let mockMouse: any; let mockCursor: any; beforeEach(() => { mockMouse = { down: jest.fn(), up: jest.fn(), move: jest.fn(), wheel: jest.fn() }; mockPage = { mouse: mockMouse, waitForTimeout: jest.fn() }; mockCursor = { moveTo: jest.fn().mockResolvedValue(undefined) }; svc = new HumanizerService(); (svc as any).createCursor = () => mockCursor; (svc as any).rand = (a: number, b: number) => (a + b) / 2; // deterministic (svc as any).chance = (_: number) => false; // never trigger random branches }); it('full 模式: ghost-cursor 移动 + 视觉停顿 + mousedown/up', async () => { await svc.click(mockPage, { x: 100, y: 200 }, 'full'); expect(mockCursor.moveTo).toHaveBeenCalledWith({ x: 100, y: 200 }, expect.any(Object)); expect(mockMouse.down).toHaveBeenCalled(); expect(mockMouse.up).toHaveBeenCalled(); }); it('fast 模式: 不走 ghost-cursor,直接 mouse.move + 短延迟', async () => { await svc.click(mockPage, { x: 100, y: 200 }, 'fast'); expect(mockCursor.moveTo).not.toHaveBeenCalled(); expect(mockMouse.move).toHaveBeenCalledWith(100, 200); expect(mockMouse.down).toHaveBeenCalled(); }); it('off 模式: 立即 click,无延迟', async () => { (svc as any).rand = jest.fn(); await svc.click(mockPage, { x: 100, y: 200 }, 'off'); expect(mockCursor.moveTo).not.toHaveBeenCalled(); expect(mockMouse.move).toHaveBeenCalledWith(100, 200); expect(mockMouse.down).toHaveBeenCalled(); expect((svc as any).rand).not.toHaveBeenCalled(); }); it('type full: 字符间随机间隔,5% 概率错字(mock 不触发)', async () => { const keyboard = { type: jest.fn(), press: jest.fn() }; mockPage.keyboard = keyboard; await svc.type(mockPage, 'hello', 'full'); // 'hello' 5 个字符 → keyboard.type 应该被调 5 次 expect(keyboard.type).toHaveBeenCalledTimes(5); expect(keyboard.type).toHaveBeenNthCalledWith(1, 'h'); }); it('type off: 直接整体输入', async () => { const keyboard = { type: jest.fn() }; mockPage.keyboard = keyboard; await svc.type(mockPage, 'hello', 'off'); expect(keyboard.type).toHaveBeenCalledWith('hello', undefined); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash pnpm jest humanizer.service ``` - [ ] **Step 3: 实现 humanizer.service.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts import { Provide, Scope, ScopeEnum } from '@midwayjs/core'; import { createCursor } from 'ghost-cursor'; export type HumanizeMode = 'full' | 'fast' | 'off'; @Provide() @Scope(ScopeEnum.Singleton) export class HumanizerService { // 注入点便于测试 protected createCursor = createCursor; protected rand(min: number, max: number): number { return min + Math.floor(Math.random() * (max - min)); } protected chance(p: number): boolean { return Math.random() < p; } async click(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise { if (mode === 'full') { // ghost-cursor 是 puppeteer 类型,patchright Page 与之不兼容,需要 as any cast const cursor = this.createCursor(page as any); await cursor.moveTo(point, { randomizeMoveDelay: true, moveDelay: this.rand(50, 150) }); await this.sleep(this.rand(100, 300)); await page.mouse.down(); await this.sleep(this.rand(50, 150)); await page.mouse.up(); if (this.chance(0.2)) { await page.mouse.wheel(0, this.rand(2, 10)); } } else if (mode === 'fast') { await page.mouse.move(point.x, point.y); await this.sleep(this.rand(50, 200)); await page.mouse.down(); await this.sleep(this.rand(20, 50)); await page.mouse.up(); } else { await page.mouse.move(point.x, point.y); await page.mouse.down(); await page.mouse.up(); } } async type(page: any, text: string, mode: HumanizeMode): Promise { if (mode === 'off') { await page.keyboard.type(text); return; } const minDelay = mode === 'full' ? 80 : 30; const maxDelay = mode === 'full' ? 250 : 80; for (let i = 0; i < text.length; i++) { const ch = text[i]; // 5% 错字(仅 full 模式) if (mode === 'full' && this.chance(0.05)) { const wrong = String.fromCharCode(ch.charCodeAt(0) + 1); await page.keyboard.type(wrong); await this.sleep(this.rand(minDelay, maxDelay)); await page.keyboard.press('Backspace'); await this.sleep(this.rand(minDelay, maxDelay)); } await page.keyboard.type(ch); await this.sleep(this.rand(minDelay, maxDelay)); } } async hover(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise { if (mode === 'full') { const cursor = this.createCursor(page as any); await cursor.moveTo(point); await this.sleep(this.rand(200, 800)); } else { await page.mouse.move(point.x, point.y); if (mode === 'fast') await this.sleep(this.rand(50, 200)); } } async scroll(page: any, deltaY: number, mode: HumanizeMode): Promise { if (mode === 'off') { await page.mouse.wheel(0, deltaY); return; } const steps = mode === 'full' ? Math.ceil(Math.abs(deltaY) / 50) : Math.ceil(Math.abs(deltaY) / 200); const stepSize = deltaY / steps; const minD = mode === 'full' ? 50 : 10; const maxD = mode === 'full' ? 150 : 30; for (let i = 0; i < steps; i++) { await page.mouse.wheel(0, stepSize); await this.sleep(this.rand(minD, maxD)); } } private sleep(ms: number): Promise { return new Promise(r => setTimeout(r, ms)); } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest humanizer.service ``` 预期:PASS(5 tests) - [ ] **Step 5: Checkpoint** `Task 8 完成`:humanizer 三档行为可用且可测。 --- ## Task 9:BrowserDaemonService.click/fill/type/snapshot/cookie/state(业务方法) **Files:** - Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts` - [ ] **Step 1: 写 SnapshotRefService 测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts import { SnapshotRefService } from '../../../../src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.js'; describe('SnapshotRefService', () => { let svc: SnapshotRefService; let mockPage: any; beforeEach(() => { svc = new SnapshotRefService(); mockPage = { evaluate: jest.fn().mockResolvedValue([ { ref: 'e1', tag: 'button', text: 'Login' }, { ref: 'e2', tag: 'input', text: '' }, ]), }; }); it('snapshot 调 page.evaluate 注入 data-ai-ref,返回 refs', async () => { const refs = await svc.snapshot('s1', mockPage); expect(refs).toEqual([ { ref: 'e1', tag: 'button', text: 'Login' }, { ref: 'e2', tag: 'input', text: '' }, ]); expect(mockPage.evaluate).toHaveBeenCalledTimes(1); }); it('refToSelector 返回 [data-ai-ref="..."]', () => { expect(svc.refToSelector('e15')).toBe('[data-ai-ref="e15"]'); }); }); ``` - [ ] **Step 2: 实现 SnapshotRefService** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts import { Provide, Scope, ScopeEnum } from '@midwayjs/core'; export interface RefEntry { ref: string; tag: string; text: string; } @Provide() @Scope(ScopeEnum.Singleton) export class SnapshotRefService { /** 给页面所有可交互元素注入 data-ai-ref,返回 ref 列表 */ async snapshot(_sessionName: string, page: any): Promise { return page.evaluate(() => { const interactive = Array.from(document.querySelectorAll( 'a, button, input, select, textarea, [role="button"], [role="link"], [contenteditable="true"]' )); const out: { ref: string; tag: string; text: string }[] = []; interactive.forEach((el, i) => { const ref = `e${i + 1}`; el.setAttribute('data-ai-ref', ref); const tag = el.tagName.toLowerCase(); let text = (el as HTMLElement).innerText || (el as HTMLInputElement).value || ''; text = text.trim().slice(0, 80); out.push({ ref, tag, text }); }); return out; }); } refToSelector(ref: string): string { return `[data-ai-ref="${ref}"]`; } } ``` - [ ] **Step 3: 跑 snapshot-ref 测试** ```bash pnpm jest snapshot-ref ``` 预期:PASS - [ ] **Step 4: 在 BrowserDaemonService 中加业务方法** 在 `daemon.service.ts` 末尾加入(在 `setHumanizeMode` 之后): ```typescript // ===== 业务方法(依赖 humanizer + snapshot-ref)===== @Inject() humanizer: HumanizerService; @Inject() snapshotRef: SnapshotRefService; async click(sessionName: string, ref: string, mode?: 'full' | 'fast' | 'off'): Promise { return this.registry.withLock(sessionName, async () => { const { page } = this.getPageOrFail(sessionName); const locator = page.locator(this.snapshotRef.refToSelector(ref)); const box = await locator.boundingBox(); if (!box) throw new Error(`Ref '${ref}' not visible`); const m = mode ?? this.getHumanizeMode(sessionName); await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m); }); } async fill(sessionName: string, ref: string, value: string, mode?: 'full' | 'fast' | 'off'): Promise { return this.registry.withLock(sessionName, async () => { const { page } = this.getPageOrFail(sessionName); const locator = page.locator(this.snapshotRef.refToSelector(ref)); const box = await locator.boundingBox(); if (!box) throw new Error(`Ref '${ref}' not visible`); const m = mode ?? this.getHumanizeMode(sessionName); await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m); await locator.fill(''); await this.humanizer.type(page, value, m); }); } async type(sessionName: string, text: string, mode?: 'full' | 'fast' | 'off'): Promise { return this.registry.withLock(sessionName, async () => { const { page } = this.getPageOrFail(sessionName); await this.humanizer.type(page, text, mode ?? this.getHumanizeMode(sessionName)); }); } async snapshot(sessionName: string) { return this.registry.withLock(sessionName, async () => { const { page } = this.getPageOrFail(sessionName); const refs = await this.snapshotRef.snapshot(sessionName, page); return { url: page.url(), refs }; }); } async getCookies(sessionName: string, domain?: string): Promise { return this.registry.withLock(sessionName, async () => { const { ctx } = this.getPageOrFail(sessionName); const cookies = await ctx.cookies(); return domain ? cookies.filter((c: any) => c.domain.includes(domain)) : cookies; }); } async saveState(sessionName: string, filePath: string): Promise { return this.registry.withLock(sessionName, async () => { const { ctx } = this.getPageOrFail(sessionName); await ctx.storageState({ path: filePath }); }); } async loadState(sessionName: string, filePath: string): Promise { // loadState 需在 launchPersistentContext 时通过 storageState 选项注入 // 但 patchright launchPersistentContext 不直接接受 storageState // → 实际方案:addCookies + 手动 localStorage(基于 file 内容) return this.registry.withLock(sessionName, async () => { const { ctx, page } = this.getPageOrFail(sessionName); const fs = await import('node:fs/promises'); const data = JSON.parse(await fs.readFile(filePath, 'utf8')); if (data.cookies) await ctx.addCookies(data.cookies); if (data.origins) { for (const o of data.origins) { await page.goto(o.origin); for (const item of o.localStorage ?? []) { await page.evaluate(([k, v]: [string, string]) => localStorage.setItem(k, v), [item.name, item.value]); } } } }); } async goto(sessionName: string, url: string): Promise { return this.registry.withLock(sessionName, async () => { const { page } = this.getPageOrFail(sessionName); await page.goto(url); }); } ``` 也要在文件顶部加 import: ```typescript import { HumanizerService } from './humanizer.service.js'; import { SnapshotRefService } from './snapshot-ref.service.js'; ``` - [ ] **Step 5: 写新方法的测试(追加到 daemon.service.test.ts)** ```typescript // 追加到 daemon.service.test.ts describe('BrowserDaemonService 业务方法', () => { let svc: BrowserDaemonService; let mockCtx: any; let mockPage: any; let mockHumanizer: any; let mockSnapshotRef: any; beforeEach(() => { mockPage = { url: () => 'https://example.com', mouse: { down: jest.fn(), up: jest.fn(), move: jest.fn() }, keyboard: { type: jest.fn() }, locator: jest.fn().mockReturnValue({ boundingBox: jest.fn().mockResolvedValue({ x: 10, y: 20, width: 100, height: 40 }), fill: jest.fn(), }), goto: jest.fn(), }; mockCtx = { pages: () => [mockPage], cookies: jest.fn().mockResolvedValue([{ name: 'a', value: '1', domain: '.x.com' }]), storageState: jest.fn(), addCookies: jest.fn(), }; mockHumanizer = { click: jest.fn(), type: jest.fn() }; mockSnapshotRef = { refToSelector: (r: string) => `[data-ai-ref="${r}"]`, snapshot: jest.fn().mockResolvedValue([{ ref: 'e1', tag: 'button', text: 'OK' }]), }; svc = new BrowserDaemonService(); const { SessionRegistry } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js'); const { SessionScheduler } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js'); (svc as any).registry = new SessionRegistry(); (svc as any).scheduler = new SessionScheduler({ maxActiveSessions: 5 }); (svc as any).humanizer = mockHumanizer; (svc as any).snapshotRef = mockSnapshotRef; (svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() }; (svc as any).registry.register('s1', mockCtx); (svc as any).scheduler.recordActivate('s1'); }); it('click: 调 humanizer.click(中心点, mode)', async () => { await svc.click('s1', 'e1'); expect(mockHumanizer.click).toHaveBeenCalledWith(mockPage, { x: 60, y: 40 }, 'full'); }); it('snapshot: 调 snapshotRef + 返回 url + refs', async () => { const r = await svc.snapshot('s1'); expect(r.url).toBe('https://example.com'); expect(r.refs).toEqual([{ ref: 'e1', tag: 'button', text: 'OK' }]); }); it('getCookies: domain 过滤', async () => { const r = await svc.getCookies('s1', 'x.com'); expect(r).toEqual([{ name: 'a', value: '1', domain: '.x.com' }]); }); it('saveState: 调 storageState({path})', async () => { await svc.saveState('s1', '/tmp/s.json'); expect(mockCtx.storageState).toHaveBeenCalledWith({ path: '/tmp/s.json' }); }); }); ``` - [ ] **Step 6: 跑所有 daemon 测试** ```bash pnpm jest daemon.service ``` 预期:PASS(前 5 tests + 4 新 tests = 9) - [ ] **Step 7: Checkpoint** `Task 9 完成`:BrowserDaemonService 业务方法齐备。 --- ## Task 9b:Cleanup + Lifecycle + Idle-Timeout(**新增 P1 修复**) **问题**:spec §5.4 要求 @Init 扫 known sessions、@Destroy 优雅关闭 + 5s 超时强制 kill;§6.6 要求 60min idle 自动回收。原 plan 未实现。 **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts` - Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` - Modify: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts` - [ ] **Step 1: 实现 cleanup.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts /** * 优雅关闭工具:等待 ctx.close() 最多 5s,超时强制 kill */ export async function gracefullyClose(ctx: any, timeoutMs = 5000): Promise { await Promise.race([ ctx.close().catch(() => {}), new Promise(resolve => setTimeout(resolve, timeoutMs)), ]); // 强制 kill 残留 chrome 进程 try { const browser = ctx.browser?.(); if (browser?.process) { const proc = browser.process(); if (proc && !proc.killed) proc.kill('SIGKILL'); } } catch {} } ``` - [ ] **Step 2: SessionScheduler 增加 idle 计时器** 修改 `session-scheduler.ts`,加入: ```typescript // 在 SessionScheduler 类内 private idleTimers = new Map(); private idleTimeoutMs: number; private onIdleTimeout?: (sessionName: string) => Promise; constructor(opts: SchedulerOptions = {}) { this.maxActiveSessions = opts.maxActiveSessions ?? 50; this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000; this.idleTimeoutMs = opts.idleTimeoutMs ?? 60 * 60 * 1000; // 60 min } setIdleTimeoutHandler(fn: (sessionName: string) => Promise): void { this.onIdleTimeout = fn; } touch(sessionName: string): void { const s = this.sessions.get(sessionName); if (s) { s.status = 'active'; s.lastUsedAt = new Date(); } this.resetIdleTimer(sessionName); } private resetIdleTimer(sessionName: string): void { this.clearIdleTimer(sessionName); if (this.idleTimeoutMs <= 0 || !this.onIdleTimeout) return; const timer = setTimeout(() => { this.idleTimers.delete(sessionName); this.onIdleTimeout?.(sessionName).catch(() => {}); }, this.idleTimeoutMs); this.idleTimers.set(sessionName, timer); } private clearIdleTimer(sessionName: string): void { const t = this.idleTimers.get(sessionName); if (t) { clearTimeout(t); this.idleTimers.delete(sessionName); } } // 在 recordActivate 末尾调 resetIdleTimer // 在 recordDeactivate 调 clearIdleTimer ``` - [ ] **Step 3: daemon.service @Init / @Destroy 钩子** 在 `daemon.service.ts` 加: ```typescript import { Init, Destroy } from '@midwayjs/core'; import { gracefullyClose } from '../runtime/cleanup.js'; import { getStateDir } from '../runtime/browser-data-dir.js'; import * as path from 'node:path'; import * as fs from 'node:fs'; // 在 BrowserDaemonService 类内: @Init() async init(): Promise { // 注册 idle 回收回调 this.scheduler.setIdleTimeoutHandler(async (sessionName) => { this.logger.info(`[browser-daemon] idle timeout: ${sessionName} → save+close`); await this.registry.withLock(sessionName, async () => { const ctx = this.registry.get(sessionName); if (!ctx) return; const stateFile = getStateDir(sessionName); try { await ctx.storageState({ path: stateFile }); } catch (e: any) { this.logger.warn(`[browser-daemon] saveState on idle failed: ${e.message}`); } await this.closeInternal(sessionName); }); }); // 扫描 known sessions(不主动 launch) const stateRoot = path.dirname(getStateDir('placeholder')); if (fs.existsSync(stateRoot)) { const files = fs.readdirSync(stateRoot).filter(f => f.endsWith('.json')); this.logger.info(`[browser-daemon] init: ${files.length} known sessions on disk`); } } @Destroy() async destroy(): Promise { this.logger.info('[browser-daemon] destroying, closing all sessions...'); const all = this.registry.list(); await Promise.all(all.map(async (e) => { try { const stateFile = getStateDir(e.sessionName); await e.context.storageState({ path: stateFile }); } catch (err: any) { this.logger.warn(`[browser-daemon] saveState ${e.sessionName} failed: ${err.message}`); } await gracefullyClose(e.context, 5000); })); } ``` - [ ] **Step 4: 写 cleanup 测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts import { gracefullyClose } from '../../../../src/modules/netaclaw/browser-daemon/runtime/cleanup.js'; describe('cleanup.gracefullyClose', () => { it('正常关闭:close 完成', async () => { const close = jest.fn().mockResolvedValue(undefined); await gracefullyClose({ close }, 1000); expect(close).toHaveBeenCalled(); }); it('close 超时:5s 后强制 kill', async () => { const kill = jest.fn(); const ctx = { close: () => new Promise(() => {}), // 永远不 resolve browser: () => ({ process: () => ({ kill, killed: false }) }), }; const start = Date.now(); await gracefullyClose(ctx, 100); expect(Date.now() - start).toBeGreaterThanOrEqual(100); expect(kill).toHaveBeenCalledWith('SIGKILL'); }); it('close 抛错不阻断', async () => { await expect( gracefullyClose({ close: () => Promise.reject(new Error('boom')) }, 100) ).resolves.toBeUndefined(); }); }); ``` - [ ] **Step 5: 跑测试** ```bash pnpm jest cleanup.test ``` 预期:PASS(3 tests) - [ ] **Step 6: SessionScheduler idle timer 测试追加** ```typescript // 追加到 session-scheduler.test.ts describe('SessionScheduler idle timer', () => { it('idle 超时后调 onIdleTimeout', async () => { const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 50 }); const handler = jest.fn().mockResolvedValue(undefined); sch.setIdleTimeoutHandler(handler); sch.recordActivate('s1'); await new Promise(r => setTimeout(r, 100)); expect(handler).toHaveBeenCalledWith('s1'); }); it('touch 重置计时器', async () => { const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 80 }); const handler = jest.fn().mockResolvedValue(undefined); sch.setIdleTimeoutHandler(handler); sch.recordActivate('s1'); await new Promise(r => setTimeout(r, 50)); sch.touch('s1'); await new Promise(r => setTimeout(r, 50)); // 累积 100ms 但 touch 重置后只过了 50ms,未触发 expect(handler).not.toHaveBeenCalled(); await new Promise(r => setTimeout(r, 50)); expect(handler).toHaveBeenCalledWith('s1'); }); }); ``` - [ ] **Step 7: Checkpoint** `Task 9b 完成`:lifecycle 钩子 + idle 自动回收 + 优雅关闭。验收 9 + 5b 可通过。 --- ## Task 10:control-auth.middleware(loopback + secret) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts` - Test: `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts` - [ ] **Step 1: 写测试** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts import { BrowserControlAuthMiddleware } from '../../../../src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.js'; describe('BrowserControlAuthMiddleware', () => { const mw = new BrowserControlAuthMiddleware(); const fn = mw.resolve(); beforeEach(() => { process.env.NETA_TRAY_SECRET = 'test-secret'; process.env.NODE_ENV = 'test'; }); it('loopback + 正确 secret → 通过', async () => { const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(next).toHaveBeenCalled(); }); it('非 loopback → 403', async () => { const ctx: any = { ip: '8.8.8.8', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(ctx.status).toBe(403); expect(next).not.toHaveBeenCalled(); }); it('loopback + 错误 secret → 401', async () => { const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'wrong' }, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(ctx.status).toBe(401); }); it('loopback 无 secret → 401', async () => { const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(ctx.status).toBe(401); }); it('dev 模式 + secret 未配置 → 旁路通过(loopback 仍校验)', async () => { process.env.NETA_TRAY_SECRET = ''; process.env.NODE_ENV = 'development'; const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(next).toHaveBeenCalled(); }); it('dev 模式 + secret 未配置 + 非 loopback → 仍 403', async () => { process.env.NETA_TRAY_SECRET = ''; process.env.NODE_ENV = 'development'; const ctx: any = { ip: '8.8.8.8', headers: {}, status: 200 }; const next = jest.fn(); await fn(ctx, next); expect(ctx.status).toBe(403); }); }); ``` - [ ] **Step 2: 跑测试看失败** ```bash pnpm jest control-auth.middleware ``` - [ ] **Step 3: 实现** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts import { IMiddleware, Middleware } from '@midwayjs/core'; import { NextFunction, Context } from '@midwayjs/koa'; import { isLoopbackAddress, validateRuntimeSecret } from '../../../../comm/runtime-secret.js'; @Middleware() export class BrowserControlAuthMiddleware implements IMiddleware { resolve() { return async (ctx: Context, next: NextFunction) => { const ip = ctx.ip || (ctx as any).request?.ip || (ctx as any).req?.socket?.remoteAddress; if (!isLoopbackAddress(ip)) { ctx.status = 403; ctx.body = { code: 1003, error: 'Forbidden: only loopback access allowed' }; return; } const expected = process.env.NETA_TRAY_SECRET ?? ''; const actual = ctx.headers['x-neta-control-secret'] as string | undefined; const isDev = process.env.NODE_ENV !== 'production'; // dev 旁路:secret 未配置时仅信任 loopback if (isDev && !expected) { await next(); return; } if (!validateRuntimeSecret(expected, actual)) { ctx.status = 401; ctx.body = { code: 1001, error: 'Unauthorized: invalid x-neta-control-secret' }; return; } await next(); }; } match(ctx: Context): boolean { return ctx.path?.startsWith('/admin/browser-daemon/'); } static getName(): string { return 'browserControlAuth'; } } ``` - [ ] **Step 4: 跑测试看通过** ```bash pnpm jest control-auth.middleware ``` 预期:PASS(6 tests) - [ ] **Step 5: 在 netaclaw/config.ts 注册中间件(重要:模块挂载明确)** 修改 `packages/backend/src/modules/netaclaw/config.ts`,把空的 `middlewares: []` 改为: ```typescript import { ModuleConfig } from '@cool-midway/core'; import { BrowserControlAuthMiddleware } from './browser-daemon/middleware/control-auth.middleware.js'; export default () => { return { name: 'NetaClaw', description: 'NetaClaw 电商浏览器自动化 Agent 引擎', middlewares: [BrowserControlAuthMiddleware], globalMiddlewares: [], order: 0, } as ModuleConfig; }; ``` > 关键:browser-daemon **不是独立模块**,是 netaclaw 的子目录。中间件挂载到 netaclaw 模块,靠 `match()` 限制只在 `/admin/browser-daemon/*` 路径生效。其他 netaclaw 路由(`/admin/netaclaw/*`)不受影响。 注意保留 NetaClawConfig / defaultNetaClawConfig 等导出(在 `// ---` 分隔符之后的部分)。 - [ ] **Step 6: Checkpoint** `Task 10 完成`:auth middleware 校验 loopback + secret,含 dev 旁路;middleware 已挂载到 netaclaw 模块。 --- ## Task 11:HTTP Controllers(5 个 controller) **Files:** - Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts` - Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts` > 这 5 个 controller 都是 thin wrapper:解析 body → 调 service → 返回结果。**严格按 Neta 现有 controller 风格**: > - `@Provide()` + `@Controller('/admin/browser-daemon')` 类装饰器声明路径前缀 > - 方法用相对路径 `@Post('/open')` `@Get('/list')` > - 直接返回 `{ code: 1000, data: ... }`,不继承 `BaseController`,不用 `this.ok()`,不用 `@CoolController` 不用 `@CoolTag` > - 参考 `packages/backend/src/modules/netaclaw/controller/admin/agent_channel.ts` 实际风格 - [ ] **Step 1: 实现 session.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts import { Provide, Inject, Controller, Post, Get, Body } from '@midwayjs/core'; import { Context } from '@midwayjs/koa'; import { BrowserDaemonService } from '../../service/daemon.service.js'; @Provide() @Controller('/admin/browser-daemon') export class AdminBrowserDaemonSessionController { @Inject() daemonService: BrowserDaemonService; @Inject() ctx: Context; @Post('/open') async open(@Body() dto: any) { try { const r = await this.daemonService.open(dto); return { code: 1000, data: r }; } catch (e: any) { this.ctx.status = e.statusCode ?? 400; return { code: 1002, error: e.message, retryAfter: e.retryAfter }; } } @Post('/close') async close(@Body('sessionName') sessionName: string) { await this.daemonService.close(sessionName); return { code: 1000, data: { ok: true } }; } @Get('/list') async list() { return { code: 1000, data: this.daemonService.list() }; } @Get('/stats') async stats() { return { code: 1000, data: this.daemonService.getStats() }; } } ``` - [ ] **Step 2: 实现 interaction.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core'; import { BrowserDaemonService } from '../../service/daemon.service.js'; @Provide() @Controller('/admin/browser-daemon') export class AdminBrowserDaemonInteractionController { @Inject() daemonService: BrowserDaemonService; @Post('/click') async click(@Body() dto: { sessionName: string; ref: string; mode?: 'full' | 'fast' | 'off' }) { await this.daemonService.click(dto.sessionName, dto.ref, dto.mode ?? 'full'); return { code: 1000, data: { clicked: dto.ref } }; } @Post('/fill') async fill(@Body() dto: { sessionName: string; ref: string; value: string; mode?: 'full' | 'fast' | 'off' }) { await this.daemonService.fill(dto.sessionName, dto.ref, dto.value, dto.mode ?? 'full'); return { code: 1000, data: { filled: dto.ref } }; } @Post('/type') async type(@Body() dto: { sessionName: string; text: string; mode?: 'full' | 'fast' | 'off' }) { await this.daemonService.type(dto.sessionName, dto.text, dto.mode ?? 'full'); return { code: 1000, data: { typed: dto.text.length } }; } } ``` - [ ] **Step 3: 实现 navigation.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core'; import { BrowserDaemonService } from '../../service/daemon.service.js'; @Provide() @Controller('/admin/browser-daemon') export class AdminBrowserDaemonNavigationController { @Inject() daemonService: BrowserDaemonService; @Post('/goto') async goto(@Body() dto: { sessionName: string; url: string }) { await this.daemonService.goto(dto.sessionName, dto.url); return { code: 1000, data: { navigated: dto.url } }; } } ``` - [ ] **Step 4: 实现 state.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core'; import { BrowserDaemonService } from '../../service/daemon.service.js'; @Provide() @Controller('/admin/browser-daemon') export class AdminBrowserDaemonStateController { @Inject() daemonService: BrowserDaemonService; @Post('/cookie-list') async cookieList(@Body() dto: { sessionName: string; domain?: string }) { return { code: 1000, data: await this.daemonService.getCookies(dto.sessionName, dto.domain) }; } @Post('/state-save') async stateSave(@Body() dto: { sessionName: string; filePath: string }) { await this.daemonService.saveState(dto.sessionName, dto.filePath); return { code: 1000, data: { saved: dto.filePath } }; } @Post('/state-load') async stateLoad(@Body() dto: { sessionName: string; filePath: string }) { await this.daemonService.loadState(dto.sessionName, dto.filePath); return { code: 1000, data: { loaded: dto.filePath } }; } } ``` - [ ] **Step 5: 实现 inspect.ts** ```typescript // packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core'; import { BrowserDaemonService } from '../../service/daemon.service.js'; @Provide() @Controller('/admin/browser-daemon') export class AdminBrowserDaemonInspectController { @Inject() daemonService: BrowserDaemonService; @Post('/snapshot') async snapshot(@Body() dto: { sessionName: string }) { return { code: 1000, data: await this.daemonService.snapshot(dto.sessionName) }; } } ``` - [ ] **Step 6: 启动 backend 验证路由** dev 模式启动(dev 旁路时无需 secret): ```bash cd packages/backend NODE_ENV=development pnpm dev ``` 另一终端: ```bash # dev 旁路(middleware 已实现):仅 loopback 即可,secret 未设也能调 curl -X GET http://localhost:8003/admin/browser-daemon/list ``` 预期:200 + `{"code":1000,"data":[]}` - [ ] **Step 7: Checkpoint** `Task 11 完成`:5 个 controller 上线,HTTP API 可达。 --- ## Task 12:Contract Test(HTTP ↔ service 等价性) **Files:** - Create: `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts` - [ ] **Step 1: 写 contract test** ```typescript // packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts import { createApp, close } from '@midwayjs/mock'; import { Framework } from '@midwayjs/koa'; describe('BrowserDaemon HTTP ↔ Service 契约', () => { let app: any; beforeAll(async () => { process.env.NETA_TRAY_SECRET = 'ct-secret'; app = await createApp(undefined, undefined, Framework); }); afterAll(async () => { if (app) await close(app); }); /** Mock daemonService,验证 controller 调它的方法名/参数与 HTTP 1:1 */ const cases = [ { http: { method: 'POST', path: '/admin/browser-daemon/close', body: { sessionName: 's1' } }, service: 'close', args: ['s1'] }, { http: { method: 'POST', path: '/admin/browser-daemon/click', body: { sessionName: 's1', ref: 'e1', mode: 'full' } }, service: 'click', args: ['s1', 'e1', 'full'] }, { http: { method: 'POST', path: '/admin/browser-daemon/fill', body: { sessionName: 's1', ref: 'e1', value: 'abc' } }, service: 'fill', args: ['s1', 'e1', 'abc', undefined] }, { http: { method: 'POST', path: '/admin/browser-daemon/cookie-list', body: { sessionName: 's1', domain: 'x.com' } }, service: 'getCookies', args: ['s1', 'x.com'] }, { http: { method: 'POST', path: '/admin/browser-daemon/snapshot', body: { sessionName: 's1' } }, service: 'snapshot', args: ['s1'] }, ]; for (const c of cases) { it(`HTTP ${c.http.method} ${c.http.path} → service.${c.service}(${c.args.join(',')})`, async () => { const fakeService = { [c.service]: jest.fn().mockResolvedValue(undefined), }; // 替换 daemonService 实例 const container = (app as any).getApplicationContext(); container.registerObject('browserDaemonService', fakeService); // 调 HTTP const supertest = require('supertest'); const res = await supertest(app.getServer())[c.http.method.toLowerCase()](c.http.path) .set('x-neta-control-secret', 'ct-secret') .send(c.http.body); expect(res.status).toBe(200); expect(fakeService[c.service]).toHaveBeenCalledWith(...c.args); }); } }); ``` > 注:实施时依据 Midway test 实际 API 调整 `registerObject` / `supertest` 接入;目的是验证 HTTP 与 service 一一对应。 - [ ] **Step 2: 跑 contract test** ```bash pnpm jest contract.test ``` 预期:PASS(5 cases) - [ ] **Step 3: Checkpoint** `Task 12 完成`:HTTP 与 service 等价性有 contract test 兜底。 --- ## Task 13-18:netabrowser-cli 包实现(合并描述) > 这 6 个 task 是 CLI 包内部各文件,模式相似(接受参数 → 调 HTTP → 输出)。每个文件按下面同一套结构实现,不再展开每 step。 ### Task 13:bin/main.ts(commander 解析 + 命令分发) **Files:** - Create: `packages/netabrowser-cli/src/bin/main.ts` - [ ] **Step 1: 实现** ```typescript #!/usr/bin/env node // packages/netabrowser-cli/src/bin/main.ts import { Command } from 'commander'; import { registerSessionCommands } from '../commands/session.js'; import { registerInteractionCommands } from '../commands/interaction.js'; import { registerNavigationCommands } from '../commands/navigation.js'; import { registerStateCommands } from '../commands/state.js'; import { registerInspectCommands } from '../commands/inspect.js'; const program = new Command(); program .name('netabrowser-cli') .description('Neta 反风控+拟人化浏览器 CLI') .version('0.1.0'); registerSessionCommands(program); registerInteractionCommands(program); registerNavigationCommands(program); registerStateCommands(program); registerInspectCommands(program); program.parseAsync(process.argv).catch(e => { console.error('Error:', e.message); process.exit(1); }); ``` - [ ] **Step 2: Checkpoint** ### Task 14:client/runtime-info + http-client **Files:** - Create: `packages/netabrowser-cli/src/client/runtime-info.ts` - Create: `packages/netabrowser-cli/src/client/http-client.ts` - [ ] **Step 1: runtime-info.ts** ```typescript // packages/netabrowser-cli/src/client/runtime-info.ts import * as fs from 'node:fs'; import * as path from 'node:path'; import * as os from 'node:os'; export interface RuntimeInfoLite { port: number; controlSecret: string; controlBaseUrl: string; } /** * v2 修复 P1-6:CLI 启动时 cwd 不一定 = backend cwd。 * runtime-info.json 真实写入 packages/backend/dist/runtime-info.json。 * 优先级:环境变量 > pkg 同目录 > monorepo backend dist > user home。 */ export function readRuntimeInfo(): RuntimeInfoLite { if (process.env.NETA_RUNTIME_INFO) { return JSON.parse(fs.readFileSync(process.env.NETA_RUNTIME_INFO, 'utf8')); } const candidates = [ // pkg 模式:netabrowser-cli.exe 同目录 data/runtime-info.json path.join(path.dirname(process.execPath), 'data', 'runtime-info.json'), // dev 模式:从 cli 编译产物上溯找 monorepo,再进 backend dist // __dirname = /packages/netabrowser-cli/dist/client // 上溯 4 层到 monorepo root,再进 packages/backend/dist path.resolve(__dirname, '../../../../../packages/backend/dist/runtime-info.json'), // 兜底:用户主目录 path.join(os.homedir(), '.neta', 'runtime-info.json'), ]; for (const p of candidates) { if (fs.existsSync(p)) return JSON.parse(fs.readFileSync(p, 'utf8')); } throw new Error( `runtime-info.json not found. Set NETA_RUNTIME_INFO env var or run backend first.\nTried:\n${candidates.map(c => ' - ' + c).join('\n')}` ); } ``` - [ ] **Step 2: http-client.ts** ```typescript // packages/netabrowser-cli/src/client/http-client.ts import axios, { AxiosInstance } from 'axios'; import { readRuntimeInfo } from './runtime-info.js'; let client: AxiosInstance | null = null; export function getHttpClient(): AxiosInstance { if (client) return client; const info = readRuntimeInfo(); client = axios.create({ baseURL: info.controlBaseUrl ?? `http://127.0.0.1:${info.port}`, headers: { 'x-neta-control-secret': info.controlSecret }, timeout: 60_000, }); return client; } export async function callDaemon(path: string, body?: any): Promise { const c = getHttpClient(); const res = await c.post(path, body ?? {}); if (res.data?.code !== 1000) { throw new Error(`Daemon error: ${res.data?.message ?? 'unknown'}`); } return res.data.data as T; } ``` - [ ] **Step 3: Checkpoint** ### Task 15:commands/session.ts(open/close/list) ```typescript // packages/netabrowser-cli/src/commands/session.ts import { Command } from 'commander'; import { callDaemon } from '../client/http-client.js'; import { formatOutput } from '../output/formatter.js'; export function registerSessionCommands(program: Command) { program .command('open ') .description('启动会话并打开 URL') .requiredOption('--session ', 'session 名称') .option('--proxy ', '代理 URL(http://user:pass@host:port)') .option('--fingerprint-seed ', '指纹 seed', parseInt) .option('--profile-dir ', 'profile 目录名') .option('--headed', '有头模式') .option('--humanize-mode ', 'full|fast|off', 'full') .option('--priority

', 'low|normal|high', 'normal') .option('--queue', '排队等位') .option('--raw', '输出 raw JSON') .action(async (url, opts) => { const dto: any = { sessionName: opts.session, url, humanizeMode: opts.humanizeMode, priority: opts.priority, queue: !!opts.queue, headed: !!opts.headed, }; if (opts.fingerprintSeed != null) dto.fingerprintSeed = opts.fingerprintSeed; if (opts.profileDir) dto.profileDir = opts.profileDir; if (opts.proxy) dto.proxy = parseProxyUrl(opts.proxy); const r = await callDaemon('/admin/browser-daemon/open', dto); console.log(formatOutput(r, opts.raw)); }); program .command('close') .requiredOption('--session ') .option('--raw') .action(async (opts) => { await callDaemon('/admin/browser-daemon/close', { sessionName: opts.session }); console.log(formatOutput({ ok: true }, opts.raw)); }); program .command('list') .description('列出所有会话') .option('--raw') .action(async (opts) => { const r = await callDaemon('/admin/browser-daemon/list'); console.log(formatOutput(r, opts.raw)); }); } function parseProxyUrl(url: string) { const u = new URL(url); return { server: `${u.protocol}//${u.host}`, username: decodeURIComponent(u.username) || undefined, password: decodeURIComponent(u.password) || undefined, }; } ``` - [ ] **Checkpoint** ### Task 16:commands/interaction.ts ```typescript // packages/netabrowser-cli/src/commands/interaction.ts import { Command } from 'commander'; import { callDaemon } from '../client/http-client.js'; import { formatOutput } from '../output/formatter.js'; export function registerInteractionCommands(program: Command) { program .command('click ') .requiredOption('--session ') .option('--mode ', 'full|fast|off') .option('--raw') .action(async (ref, opts) => { // v2 修复 P1-8:CLI 端 mode 优先级 = 命令级 --mode > NETA_BROWSER_HUMANIZE_MODE 环境变量 > 默认 full const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full'; await callDaemon('/admin/browser-daemon/click', { sessionName: opts.session, ref, mode }); console.log(formatOutput({ clicked: ref }, opts.raw)); }); program .command('fill ') .requiredOption('--session ') .option('--mode ') .option('--raw') .action(async (ref, value, opts) => { const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full'; await callDaemon('/admin/browser-daemon/fill', { sessionName: opts.session, ref, value, mode }); console.log(formatOutput({ filled: ref }, opts.raw)); }); program .command('type ') .requiredOption('--session ') .option('--mode ') .option('--raw') .action(async (text, opts) => { const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full'; await callDaemon('/admin/browser-daemon/type', { sessionName: opts.session, text, mode }); console.log(formatOutput({ typed: text.length }, opts.raw)); }); } ``` - [ ] **Checkpoint** ### Task 17:commands/navigation + state + inspect ```typescript // packages/netabrowser-cli/src/commands/navigation.ts import { Command } from 'commander'; import { callDaemon } from '../client/http-client.js'; import { formatOutput } from '../output/formatter.js'; export function registerNavigationCommands(program: Command) { program .command('goto ') .requiredOption('--session ') .option('--raw') .action(async (url, opts) => { await callDaemon('/admin/browser-daemon/goto', { sessionName: opts.session, url }); console.log(formatOutput({ navigated: url }, opts.raw)); }); } ``` ```typescript // packages/netabrowser-cli/src/commands/state.ts import { Command } from 'commander'; import { callDaemon } from '../client/http-client.js'; import { formatOutput } from '../output/formatter.js'; export function registerStateCommands(program: Command) { program .command('cookie-list') .requiredOption('--session ') .option('--domain ') .option('--raw') .action(async (opts) => { const r = await callDaemon('/admin/browser-daemon/cookie-list', { sessionName: opts.session, domain: opts.domain }); console.log(formatOutput(r, opts.raw)); }); program .command('state-save') .requiredOption('--session ') .requiredOption('--output ') .option('--raw') .action(async (opts) => { await callDaemon('/admin/browser-daemon/state-save', { sessionName: opts.session, filePath: opts.output }); console.log(formatOutput({ saved: opts.output }, opts.raw)); }); program .command('state-load') .requiredOption('--session ') .requiredOption('--input ') .option('--raw') .action(async (opts) => { await callDaemon('/admin/browser-daemon/state-load', { sessionName: opts.session, filePath: opts.input }); console.log(formatOutput({ loaded: opts.input }, opts.raw)); }); } ``` ```typescript // packages/netabrowser-cli/src/commands/inspect.ts import { Command } from 'commander'; import { callDaemon } from '../client/http-client.js'; import { formatOutput } from '../output/formatter.js'; export function registerInspectCommands(program: Command) { program .command('snapshot') .requiredOption('--session ') .option('--raw') .action(async (opts) => { const r = await callDaemon('/admin/browser-daemon/snapshot', { sessionName: opts.session }); console.log(formatOutput(r, opts.raw)); }); } ``` - [ ] **Checkpoint** ### Task 18:output/formatter.ts ```typescript // packages/netabrowser-cli/src/output/formatter.ts export function formatOutput(data: any, raw?: boolean): string { if (raw) return JSON.stringify(data); if (Array.isArray(data) && data.length > 0 && data[0]?.ref) { // snapshot 友好输出 return data.map((r: any) => `[${r.ref}] ${r.tag}: ${r.text}`).join('\n'); } return JSON.stringify(data, null, 2); } ``` 写最小测试: ```typescript // packages/netabrowser-cli/tests/output-formatter.test.ts import { formatOutput } from '../src/output/formatter.js'; describe('formatter', () => { it('--raw → 单行 JSON', () => { expect(formatOutput({ a: 1 }, true)).toBe('{"a":1}'); }); it('snapshot refs → 友好输出', () => { const r = formatOutput([{ ref: 'e1', tag: 'button', text: 'OK' }]); expect(r).toContain('[e1] button: OK'); }); }); ``` - [ ] **Step:Checkpoint** --- ## Task 19:SKILL.md(NetaClaw Agent skill 元数据) **Files:** - Create: `packages/backend/skills/netabrowser-cli/SKILL.md` ```markdown --- name: netabrowser-cli description: Neta 反风控+拟人化浏览器自动化 CLI。养号、电商自动化、AI 探索复杂网页时使用。比 playwright-cli 更难被反风控识别(fingerprint-chromium 内核 + patchright 反自动化 + ghost-cursor 拟人化轨迹)。100 账号矩阵养号场景首选。 allowed-tools: Bash(netabrowser-cli:*) Bash(npx:*) --- # Browser Automation with netabrowser-cli ## When to use vs playwright-cli vs patchwright-cli - **netabrowser-cli**:国内反风控严的站(小红书/抖音/淘宝/拼多多/微博)+ 多账号矩阵 + 养号场景 - **patchwright-cli**:国外 Cloudflare/DataDome 等反爬保护的站 - **playwright-cli**:测试、无反风控的站 If targeting Chinese social/e-commerce sites or running multi-account automation, **prefer netabrowser-cli**. ## Quick start ```bash # 启动会话(每会话独立指纹+独立 IP) netabrowser-cli open https://www.xiaohongshu.com \ --session=acc1 \ --proxy=http://user:pass@host:port \ --fingerprint-seed=12345 \ --headed # 拿当前页面可交互元素 netabrowser-cli snapshot --session=acc1 # 输出:[e1] button: 登录 / [e2] input: 手机号 / ... # 拟人化点击 + 输入(默认 humanize-mode=full) netabrowser-cli click e1 --session=acc1 netabrowser-cli fill e2 13800001234 --session=acc1 # 抓 cookie netabrowser-cli cookie-list --session=acc1 --domain=xiaohongshu.com # 保存登录态 netabrowser-cli state-save --session=acc1 --output=/path/to/state.json # 关闭 netabrowser-cli close --session=acc1 ``` ## Commands ### Session 管理 - `open ` 启动会话 - `close` 关闭 - `list` 列出所有 session ### 交互(默认拟人化 full) - `click ` 点击 ref 元素(贝塞尔轨迹+视觉停顿+mousedown/up) - `fill ` focus + 拟人化输入 - `type ` 在已 focused 输入框打字 - `--mode=fast` 单命令切批量模式(200-500ms 延迟,无轨迹) - `--mode=off` 测试用,立即执行 ### 导航 - `goto ` 跳转 ### 状态 - `cookie-list [--domain=x]` - `state-save --output=path` 保存完整登录态(cookie+localStorage) - `state-load --input=path` 恢复 ### 检查 - `snapshot` 拿可交互元素 ref 列表 ## 参数规则 ### 指纹 - `--fingerprint-seed=N` 单一 seed 派生所有指纹维度(推荐) - 不同 seed → 不同 canvas/webgl/UA fingerprint,同 seed → 完全可复现 ### 代理 - `--proxy=http://user:pass@host:port` 单参数最方便 - 出口 IP 实测 = 代理 IP(patchright `launchPersistentContext` 验证通过) ### 拟人化档位 | `--humanize-mode` | 单命令开销 | 适用 | |---|---|---| | full(默认) | 2-5s | 养号、敏感操作、AI 探索 | | fast | 0.3-0.7s | 批量发布、批量评论 | | off | <100ms | 测试、CI | ### 调度 - `--priority=low\|normal\|high` 优先级 - `--queue` 触达上限时排队(最长 30s) ## 性能特征(重要) 养号场景每号约 30 操作 × 5s ≈ 2.5min。100 号串行 ≈ 4 小时。批量发布请用 `--humanize-mode=fast`。 ## 详细文档 - [拟人化档位详解](references/humanization.md) - [指纹参数](references/fingerprint.md) - [代理配置](references/proxy.md) - [典型场景示例](references/examples.md) ``` - [ ] **Step:Checkpoint** ## Task 20:references/ 子文档 > 创建 4 个 reference 文件作为 SKILL.md 的扩展。每个文件 50-100 行,按 SKILL.md 中链接对应的主题展开。 **Files:** - Create: `packages/backend/skills/netabrowser-cli/references/humanization.md` - Create: `packages/backend/skills/netabrowser-cli/references/fingerprint.md` - Create: `packages/backend/skills/netabrowser-cli/references/proxy.md` - Create: `packages/backend/skills/netabrowser-cli/references/examples.md` 每个文件简明阐述对应主题,含示例命令。具体内容由实施者从 spec §6.1-6.6 + 实测脚本提炼。 - [ ] **Step:Checkpoint** --- ## Task 21:Windows 打包集成 **Files:** - Modify: `packages/backend/scripts/pkg-build.js` - Modify: `packages/backend/scripts/build-windows-installer.js` - Modify: `packages/backend/installer/setup.iss` - [ ] **Step 1: pkg-build.js 增加 netabrowser-cli 编译** 读 `pkg-build.js` 现有逻辑,在 backend.exe 编译之后增加: ```javascript // 编译 netabrowser-cli console.log('Building netabrowser-cli...'); const cliRoot = path.resolve(__dirname, '../../netabrowser-cli'); execSync('pnpm build', { cwd: cliRoot, stdio: 'inherit' }); // pkg 打包 const pkgConfig = { name: 'netabrowser-cli', bin: path.join(cliRoot, 'dist', 'bin', 'main.js'), targets: ['node22-win-x64'], outputPath: path.join(__dirname, '../installer/dist/netabrowser-cli.exe'), }; await pkg(pkgConfig); ``` - [ ] **Step 2: setup.iss 增加 [Files] 段** 在现有 `[Files]` 后追加: ```iss Source: "dist/netabrowser-cli.exe"; DestDir: "{app}"; Flags: ignoreversion Source: "..\..\netabrowser-cli\chromium\win64\*"; DestDir: "{app}\chromium\win64"; Flags: ignoreversion recursesubdirs createallsubdirs Source: "..\..\backend\skills\netabrowser-cli\*"; DestDir: "{app}\skills\netabrowser-cli"; Flags: ignoreversion recursesubdirs createallsubdirs ``` - [ ] **Step 3: build-windows-installer.js 加 chromium 复制步骤** 在调 ISCC 之前确保 `chromium/win64` 存在。 - [ ] **Step 4: 实跑一次安装包构建(dev 机器)** ```bash cd packages/backend pnpm run build:windows-installer ``` 预期:生成的 setup.exe 大小 ~250-400MB。装到测试 Windows,能找到 `C:\Program Files\Neta\netabrowser-cli.exe` 和 `chromium/win64/chrome.exe`。 - [ ] **Step 5: Checkpoint** --- ## Task 22:联调与冒烟(手工验收) 按 spec §9 验收清单逐条跑: - [ ] **Step 1: 跑所有单测** ```bash cd packages/backend pnpm jest test/modules/netaclaw/browser-daemon/ ``` 预期:全 PASS - [ ] **Step 2: 启动 backend dev 模式** ```bash NODE_ENV=development pnpm dev ``` 预期:runtime-info.json 写入 `packages/backend/dist/runtime-info.json` 含 controlSecret(dev 模式下 secret 可能为空字符串,middleware 已开 dev 旁路) > v2 注:dev 模式下 NETA_TRAY_SECRET 通常未设。middleware 已实现 dev 旁路:`NODE_ENV !== 'production' && expected === ''` → 仅校验 loopback、跳过 secret 校验。 - [ ] **Step 3: cli list 应返回空** ```bash node packages/netabrowser-cli/dist/bin/main.js list --raw ``` 预期:`[]` - [ ] **Step 4: 真代理出口 IP 验证** ```bash node packages/netabrowser-cli/dist/bin/main.js open https://httpbin.org/ip \ --session=test \ --proxy="http://e50b26:7ecdccfd@210.51.27.112:10000" \ --headed ``` 然后: ```bash node packages/netabrowser-cli/dist/bin/main.js snapshot --session=test ``` 看 url + 内容含 `"origin": "210.51.27.112"` - [ ] **Step 5: creepjs 反检测 trust score** 打开 https://abrahamjuliot.github.io/creepjs/,肉眼记 trust score。预期 ≥ 70%。 - [ ] **Step 6: 拟人化档位延迟测量** ```bash time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=full time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=fast time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=off ``` 预期:full ~2-5s, fast ~0.3-0.7s, off <0.2s - [ ] **Step 7: HTTP 直接调,验证 auth** **dev 旁路验证**: ```bash # dev 模式(NODE_ENV=development,NETA_TRAY_SECRET 未设) curl http://localhost:8003/admin/browser-daemon/list # 200(dev 旁路允许 loopback) # 非 loopback 仍 403 curl --interface 8.8.8.8 http://localhost:8003/admin/browser-daemon/list # 403 ``` **prod 风格验证**: ```bash # 杀掉 backend,重新启动 prod 模式 NODE_ENV=production NETA_TRAY_SECRET=test-prod-secret pnpm start # 无 secret → 401 curl http://localhost:8003/admin/browser-daemon/list # 401 # 错误 secret → 401 curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: wrong" # 401 # 正确 secret → 200 curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: test-prod-secret" # 200 ``` - [ ] **Step 8: 软上限验证** 设 `maxActiveSessions=2`,启动 3 个 session 看第 3 个返回 503。 - [ ] **Step 9: backend 重启 ref 失效** ctrl+C 关 backend,重启 `pnpm dev`,调 `click e1 --session=test`。 预期:报错(session 不存在或 ref 失效),需要先 `open` 再 `snapshot` 再 `click`。 - [ ] **Step 10: 安装包冒烟** 把 setup.exe 装到全新 Windows,启动托盘 → 后端启动 → cli 跑 step 4-5。 - [ ] **Step 11: Checkpoint** `Task 22 完成`:所有验收标准通过。 --- ## Task 23:文档更新 + 路线图 **Files:** - Modify: `docs/superpowers/specs/2026-05-04-netabrowser-cli-s1-design.md`(变更日志) - Modify: `docs/superpowers/specs/2026-05-03-geo-master-roadmap.md`(如存在则更新) - [ ] **Step 1: 更新 spec §13 变更日志** 加一行:"2026-XX-XX 实施完成,所有验收标准通过,准备进入 S2"。 - [ ] **Step 2: 更新 geo 路线图** 如有 geo master-roadmap,标注 netabrowser-cli S1 完成 → 解锁 geo BrowserAutomationService 迁移。 - [ ] **Step 3: 提示 user 走完整 git commit** ```bash cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo git status # 用户审核后统一 commit(如其要求) ``` - [ ] **Step 4: Checkpoint** `Task 23 完成`:文档同步。等待 user 触发统一 commit。 --- ## 实施期间约束 - ❌ 不写 SQL 文件(如果有数据库需求,用 mcp mysql 工具直接 INSERT) - ❌ 每个 task **不单独 git commit**(用户要求全部完成 + 联调通过后统一提交) - ❌ 不修改现有 playwright-cli / patchwright-cli skill - ❌ 不在 controller 中加业务逻辑(只能是 thin wrapper,§5.3.1 强制) - ❌ HTTP API 必须强制 loopback + secret,不能例外 - ❌ 不实现平台特定补丁(小红书/抖音/淘宝等)→ S2 --- ## 实施完成后 - 调用 `superpowers:verification-before-completion` 逐条核对验收标准 - 调用 `superpowers:requesting-code-review` 触发代码审查 - 用户测试通过后统一 git commit - 更新 spec §13 变更日志 - 进入下一个子项目:geo BrowserAutomationService 迁移到 BrowserDaemonService(S2)