3174 lines
108 KiB
Markdown
3174 lines
108 KiB
Markdown
|
|
# netabrowser-cli S1 Implementation Plan
|
|||
|
|
|
|||
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|||
|
|
>
|
|||
|
|
> **上位文档**:[`../specs/2026-05-04-netabrowser-cli-s1-design.md`](../specs/2026-05-04-netabrowser-cli-s1-design.md)
|
|||
|
|
> **Git 策略**:用户要求"全部完成 + 联调通过后统一提交"。每 task 末尾标注 `Checkpoint`(仅作进度标记,不做 git commit)。
|
|||
|
|
|
|||
|
|
**Goal:** 在 Neta monorepo 内交付 netabrowser-cli S1 基础设施:BrowserDaemonService(嵌入 backend)+ netabrowser-cli 二进制 + skill 元数据,提供反风控+拟人化的浏览器自动化能力,统一服务于 NetaClaw Agent 探索期(CLI)和后端业务固化期(@Inject service)。
|
|||
|
|
|
|||
|
|
**Architecture:** Service-First 架构:核心是 backend 内的 `BrowserDaemonService`(Midway @Singleton)持有 `Map<sessionName, BrowserContext>`;CLI 是 thin client 通过 HTTP loopback + secret 调 backend;底层用 `patchright` (npm) 启动 vendored `neta-chromium` (fingerprint-chromium);拟人化用 `ghost-cursor` + 自加套,分 full/fast/off 三档;session-name 串行化锁;LRU 软上限调度。
|
|||
|
|
|
|||
|
|
**Tech Stack:** Midway.js 3.20 / TypeScript 5.9 / `patchright` (npm) / `ghost-cursor` (npm) / fingerprint-chromium 144.0.7559.132 (vendored 二进制) / commander (CLI 解析) / jest+ts-jest / Inno Setup / yao-pkg
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚠️ v2 修复总览(架构 review 后必读)
|
|||
|
|
|
|||
|
|
本 plan 经过架构师交叉验证发现 8 项 P0/P1 问题,已就地修复。实施时优先理解这些修复点:
|
|||
|
|
|
|||
|
|
| # | 问题 | 修复落点 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| **P0-1** | Controller 路径与 Cool Admin 风格冲突 | Task 11 全部 controller 改用 `@Provide()` + `@Controller('/admin/browser-daemon/<sub>')` + 方法相对路径,参考 `netaclaw/controller/admin/agent_channel.ts` 风格 |
|
|||
|
|
| **P0-2** | dev 模式 dataDir = `<cwd>/dist`,build 清空状态 | 新增 Task 4b:`resolveBrowserDataDir()` 独立解析(dev 模式 → `<monorepo>/.netabrowser-data/`,prod → `<dataDir>/.netabrowser/`) |
|
|||
|
|
| **P0-3** | browser-daemon 模块挂载错位 | Task 10 Step 5 明确:把 `BrowserControlAuthMiddleware` 加到 `netaclaw/config.ts` 的 `middlewares` 数组,依靠 `match()` 仅作用于 `/admin/browser-daemon/*` 路径 |
|
|||
|
|
| **P0-4** | ghost-cursor 类型与 patchright 不兼容 | Task 8 Step 3 humanizer 内部 `createCursor(page as any)` cast |
|
|||
|
|
| **P1-5** | dev 模式 NETA_TRAY_SECRET 未设 → 全 401 | Task 10 middleware 加 dev 旁路:`process.env.NODE_ENV !== 'production' && expected === ''` 时跳过 secret 校验 |
|
|||
|
|
| **P1-6** | CLI runtime-info 路径错位(cwd 不对) | Task 14 候选改为 `<monorepo-root>/packages/backend/dist/runtime-info.json` + 强制 `NETA_RUNTIME_INFO` 环境变量优先 |
|
|||
|
|
| **P1-7** | @Init/@Destroy lifecycle + 60min idle 自动回收缺失 | 新增 Task 9b:`runtime/cleanup.ts` + daemon.service `@Init`/`@Destroy` + scheduler idle timer |
|
|||
|
|
| **P1-8** | CLI 与会话级 mode 协同 bug | service 移除 modeMap(service-stateless),CLI 端用环境变量 `NETA_BROWSER_HUMANIZE_MODE` 作"会话级默认",命令级 `--mode` 覆盖 |
|
|||
|
|
|
|||
|
|
**额外路径修正**:Task 4 `chromium-launcher` `__dirname` 上溯 **7 层**(不是 6 层)—— 实际路径 `dist/modules/netaclaw/browser-daemon/runtime/` 到 monorepo root 是 7 个 `..`。
|
|||
|
|
|
|||
|
|
**版本锁定**:Task 1 安装命令改为 `pnpm add patchright@1.59.4 ghost-cursor@1.4.2`(已验证兼容版本)。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 文件结构
|
|||
|
|
|
|||
|
|
### 后端模块(新增 14 个文件)
|
|||
|
|
|
|||
|
|
| 路径 | 职责 |
|
|||
|
|
|---|---|
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/config.ts` | 模块配置 + `maxActiveSessions` 等 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts` | ★ BrowserDaemonService 核心,统一锁 + 调度入口 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts` | 拟人化包装(ghost-cursor + 三档) |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts` | neta-chromium 指纹参数生成 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts` | AI ref 协议(spike #1 选定方案) |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts` | 路径解析 + patchright launch 包装 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts` | Map<name, context> + 串行化锁 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts` | LRU 软上限调度 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts` | @Destroy 时优雅关闭 |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts` | loopback + secret auth |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts` | /open /close /list /stats |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts` | /click /fill /type /scroll /hover /press |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts` | /goto /back /forward /reload |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts` | /cookie-list /cookie-set /state-save /state-load |
|
|||
|
|
| `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts` | /snapshot /screenshot /eval /run-code |
|
|||
|
|
|
|||
|
|
### netabrowser-cli 包(新增 13 个文件)
|
|||
|
|
|
|||
|
|
| 路径 | 职责 |
|
|||
|
|
|---|---|
|
|||
|
|
| `packages/netabrowser-cli/package.json` | bin 声明 + patchright/commander 等依赖 |
|
|||
|
|
| `packages/netabrowser-cli/tsconfig.json` | TS 配置 |
|
|||
|
|
| `packages/netabrowser-cli/src/bin/main.ts` | CLI 入口(commander 解析) |
|
|||
|
|
| `packages/netabrowser-cli/src/client/runtime-info.ts` | 读 backend runtime-info.json |
|
|||
|
|
| `packages/netabrowser-cli/src/client/http-client.ts` | HTTP 请求 + secret header |
|
|||
|
|
| `packages/netabrowser-cli/src/output/formatter.ts` | --raw vs 默认 + 错误格式化 |
|
|||
|
|
| `packages/netabrowser-cli/src/commands/session.ts` | open/close/list(共用一个文件) |
|
|||
|
|
| `packages/netabrowser-cli/src/commands/interaction.ts` | click/fill/type/scroll/hover/press |
|
|||
|
|
| `packages/netabrowser-cli/src/commands/navigation.ts` | goto/back/forward/reload |
|
|||
|
|
| `packages/netabrowser-cli/src/commands/state.ts` | cookie / state |
|
|||
|
|
| `packages/netabrowser-cli/src/commands/inspect.ts` | snapshot/screenshot/eval/run-code |
|
|||
|
|
|
|||
|
|
### 测试文件(新增 11 个)
|
|||
|
|
|
|||
|
|
| 路径 | 测试对象 |
|
|||
|
|
|---|---|
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts` | 路径解析 + launch args 拼装 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts` | seed → args 派生 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts` | 三档行为 + 注入 random source 验证延迟范围 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts` | 串行化锁竞态 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts` | LRU + 软上限 + 队列 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts` | ref 协议契约 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts` | 编排:open/click/cookie/close mock 全链路 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts` | loopback + secret 校验 |
|
|||
|
|
| `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts` | HTTP ↔ service 等价性 contract |
|
|||
|
|
| `packages/netabrowser-cli/tests/cli-parse.test.ts` | commander 参数解析 |
|
|||
|
|
| `packages/netabrowser-cli/tests/output-formatter.test.ts` | 输出格式 |
|
|||
|
|
|
|||
|
|
### Skill 元数据(新增 5 个文件)
|
|||
|
|
|
|||
|
|
| 路径 | 职责 |
|
|||
|
|
|---|---|
|
|||
|
|
| `packages/backend/skills/netabrowser-cli/SKILL.md` | 给 NetaClaw Agent 看的命令清单 |
|
|||
|
|
| `packages/backend/skills/netabrowser-cli/references/humanization.md` | 三档详解 |
|
|||
|
|
| `packages/backend/skills/netabrowser-cli/references/fingerprint.md` | 指纹参数 |
|
|||
|
|
| `packages/backend/skills/netabrowser-cli/references/proxy.md` | 代理配置 |
|
|||
|
|
| `packages/backend/skills/netabrowser-cli/references/examples.md` | 典型场景 |
|
|||
|
|
|
|||
|
|
### 集成改动
|
|||
|
|
|
|||
|
|
| 路径 | 改动 |
|
|||
|
|
|---|---|
|
|||
|
|
| `pnpm-workspace.yaml` | 新增 `packages/netabrowser-cli` |
|
|||
|
|
| `packages/backend/package.json` | 新增 `patchright` + `ghost-cursor` 依赖 |
|
|||
|
|
| `packages/backend/installer/setup.iss` | `[Files]` 段加 chromium/win64 + netabrowser-cli.exe |
|
|||
|
|
| `packages/backend/scripts/build-windows-installer.js` | 复制 chromium 二进制步骤 |
|
|||
|
|
| `packages/backend/scripts/pkg-build.js` | 增加 netabrowser-cli pkg 编译 |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 实施依赖图
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Phase 0: T0(包脚手架) → T1(npm 依赖)
|
|||
|
|
↓
|
|||
|
|
Phase 1: T2(Spike #1 ref 协议) ‖ T3(Spike #2 ghost-cursor 兼容) ← 阻塞门
|
|||
|
|
↓
|
|||
|
|
Phase 2: T4(launcher) → T5(registry+lock) → T6(scheduler) → T7(open/close)
|
|||
|
|
↓
|
|||
|
|
T8(humanizer + click/fill) → T9(state/cookie/snapshot)
|
|||
|
|
↓
|
|||
|
|
Phase 3: T10(auth middleware) → T11(controllers) → T12(contract test)
|
|||
|
|
↓
|
|||
|
|
Phase 4: T13(cli bin) → T14(client) → T15-17(命令实现) → T18(formatter)
|
|||
|
|
↓
|
|||
|
|
Phase 5: T19-20(skill 元数据)
|
|||
|
|
↓
|
|||
|
|
Phase 6: T21(pkg build) → T22(installer) → T23(路径解析)
|
|||
|
|
↓
|
|||
|
|
Phase 7: T24(联调冒烟) → T25(性能验证) → T26(文档)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 0:netabrowser-cli 包脚手架
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/netabrowser-cli/package.json`
|
|||
|
|
- Create: `packages/netabrowser-cli/tsconfig.json`
|
|||
|
|
- Modify: `pnpm-workspace.yaml`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 把 netabrowser-cli 加入 pnpm workspace**
|
|||
|
|
|
|||
|
|
读 `pnpm-workspace.yaml`,确认含有 `packages/*` 通配符(应该已有)。如无 `packages/*` 通配符,加上:
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
packages:
|
|||
|
|
- 'packages/*'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
无需修改的话跳过。
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 创建 netabrowser-cli/package.json**
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
{
|
|||
|
|
"name": "@neta/netabrowser-cli",
|
|||
|
|
"version": "0.1.0",
|
|||
|
|
"description": "Neta 反风控+拟人化浏览器自动化 CLI",
|
|||
|
|
"private": true,
|
|||
|
|
"type": "module",
|
|||
|
|
"bin": {
|
|||
|
|
"netabrowser-cli": "./dist/bin/main.js"
|
|||
|
|
},
|
|||
|
|
"scripts": {
|
|||
|
|
"build": "tsc -p tsconfig.json",
|
|||
|
|
"test": "jest"
|
|||
|
|
},
|
|||
|
|
"dependencies": {
|
|||
|
|
"commander": "^12.0.0",
|
|||
|
|
"axios": "^1.7.0"
|
|||
|
|
},
|
|||
|
|
"devDependencies": {
|
|||
|
|
"@types/node": "^22.0.0",
|
|||
|
|
"jest": "^29.7.0",
|
|||
|
|
"ts-jest": "^29.1.5",
|
|||
|
|
"typescript": "^5.9.0"
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 创建 netabrowser-cli/tsconfig.json**
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
{
|
|||
|
|
"compilerOptions": {
|
|||
|
|
"target": "ES2022",
|
|||
|
|
"module": "NodeNext",
|
|||
|
|
"moduleResolution": "NodeNext",
|
|||
|
|
"outDir": "dist",
|
|||
|
|
"rootDir": "src",
|
|||
|
|
"strict": true,
|
|||
|
|
"esModuleInterop": true,
|
|||
|
|
"skipLibCheck": true,
|
|||
|
|
"resolveJsonModule": true,
|
|||
|
|
"declaration": false
|
|||
|
|
},
|
|||
|
|
"include": ["src/**/*"]
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 验证 pnpm 识别 workspace**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
|
|||
|
|
pnpm install
|
|||
|
|
pnpm list -r --depth=-1 | grep netabrowser-cli
|
|||
|
|
```
|
|||
|
|
预期:列表中出现 `@neta/netabrowser-cli`
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 0 完成`:包注册到 workspace,可以独立 build/test。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 1:backend 安装 patchright + ghost-cursor 依赖
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Modify: `packages/backend/package.json`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 在 backend 安装 patchright(npm 包,锁定版本)**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
pnpm add patchright@1.59.4
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> 锁定 1.59.4:经 review 验证此版本兼容 Node 22 + Midway 3.20 + neta-chromium 144.0.7559.132。后续升级前需重测兼容性。
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 安装 ghost-cursor(锁定版本)**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm add ghost-cursor@1.4.2
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 验证 patchright 能 import**
|
|||
|
|
|
|||
|
|
跑一段临时脚本:
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
node -e "import('patchright').then(p => console.log('patchright loaded:', Object.keys(p).join(',')))"
|
|||
|
|
```
|
|||
|
|
预期输出含 `chromium,firefox,webkit,...`
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 验证 ghost-cursor 能 import**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
node -e "import('ghost-cursor').then(p => console.log('ghost-cursor loaded:', Object.keys(p).join(',')))"
|
|||
|
|
```
|
|||
|
|
预期含 `createCursor,path,...`
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 1 完成`:依赖装好,可以在 service 代码 import 使用。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 2 [SPIKE #1]:AI Ref 协议方案验证
|
|||
|
|
|
|||
|
|
> 这是 spec §11 的阻塞门 spike。**必须先做,否则后续 interaction 命令无法实现**。
|
|||
|
|
|
|||
|
|
**目标**:选定一种 AI ref 协议方案并 demo 跑通。
|
|||
|
|
|
|||
|
|
候选方案:
|
|||
|
|
- **A. 自实现**:snapshot 时给 DOM 注入 `data-ai-ref="e15"` 属性 + 内存维护 `Map<sessionName, Map<ref, ElementHandle>>`
|
|||
|
|
- **B. 改用 selector**:命令直接接收 selector 字符串(`click 'button:has-text("登录")'`),不要 ref
|
|||
|
|
- **C. vendor playwright-cli ref 代码**:剥离 playwright-cli 内部 ref 实现引入 netabrowser-cli
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写 spike 测试(方案 A,先验证最简实现)**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts
|
|||
|
|
import { chromium } from 'patchright';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import * as fs from 'node:fs';
|
|||
|
|
|
|||
|
|
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
|
|||
|
|
|
|||
|
|
describe('Spike #1: AI ref 协议(方案 A 自实现)', () => {
|
|||
|
|
jest.setTimeout(60000);
|
|||
|
|
|
|||
|
|
it('A: snapshot 注入 data-ai-ref,可基于 ref 定位元素', async () => {
|
|||
|
|
if (!fs.existsSync(CHROME)) {
|
|||
|
|
console.warn('SKIP: neta-chromium not found at', CHROME);
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-1');
|
|||
|
|
const ctx = await chromium.launchPersistentContext(tmpProfile, {
|
|||
|
|
executablePath: CHROME,
|
|||
|
|
headless: true,
|
|||
|
|
args: ['--fingerprint=12345'],
|
|||
|
|
});
|
|||
|
|
const page = ctx.pages()[0] || (await ctx.newPage());
|
|||
|
|
await page.goto('data:text/html,<button id=login>Login</button><a href=#>Link</a>');
|
|||
|
|
|
|||
|
|
// 1. snapshot:给所有可交互元素注入 data-ai-ref
|
|||
|
|
const refs = await page.evaluate(() => {
|
|||
|
|
const interactive = Array.from(document.querySelectorAll('button, a, input, select, textarea'));
|
|||
|
|
const out: { ref: string; tag: string; text: string }[] = [];
|
|||
|
|
interactive.forEach((el, i) => {
|
|||
|
|
const ref = `e${i + 1}`;
|
|||
|
|
el.setAttribute('data-ai-ref', ref);
|
|||
|
|
out.push({ ref, tag: el.tagName.toLowerCase(), text: (el.textContent || '').trim() });
|
|||
|
|
});
|
|||
|
|
return out;
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
expect(refs).toEqual([
|
|||
|
|
{ ref: 'e1', tag: 'button', text: 'Login' },
|
|||
|
|
{ ref: 'e2', tag: 'a', text: 'Link' },
|
|||
|
|
]);
|
|||
|
|
|
|||
|
|
// 2. 通过 ref click
|
|||
|
|
await page.locator('[data-ai-ref="e1"]').click();
|
|||
|
|
// 不报错 = 通过
|
|||
|
|
|
|||
|
|
await ctx.close();
|
|||
|
|
fs.rmSync(tmpProfile, { recursive: true, force: true });
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑 spike 测试**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
pnpm jest test/modules/netaclaw/browser-daemon/spike1-ref-protocol.test.ts -t 'A:'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
预期:PASS(含 fingerprint chromium 已在 vendor 里)
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 评审 spike 结果**
|
|||
|
|
|
|||
|
|
如果方案 A PASS,写入决策文件:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cat > packages/backend/src/modules/netaclaw/browser-daemon/SPIKE_DECISION.md << 'EOF'
|
|||
|
|
# Spike #1 决策:AI Ref 协议
|
|||
|
|
|
|||
|
|
**选定方案 A:自实现 data-ai-ref 注入**
|
|||
|
|
|
|||
|
|
## 理由
|
|||
|
|
- 实现简单(一段 page.evaluate JS 即可)
|
|||
|
|
- 不引入额外依赖
|
|||
|
|
- 完全可控
|
|||
|
|
|
|||
|
|
## 实现要点
|
|||
|
|
1. snapshot 命令时调用 `injectRefs(page)`,给所有可交互元素加 `data-ai-ref="eN"`
|
|||
|
|
2. service 内存维护 `Map<sessionName, Map<ref, locator-info>>` 用于审计/调试
|
|||
|
|
3. interaction 命令收到 ref 后用 `page.locator('[data-ai-ref="${ref}"]')` 定位
|
|||
|
|
4. 页面跳转/重新 snapshot 后旧 ref 失效(DOM 变化)
|
|||
|
|
|
|||
|
|
## 不选 B/C 的理由
|
|||
|
|
- B(selector):AI 写 selector 容易错;ref 抽象更友好
|
|||
|
|
- C(vendor playwright-cli):playwright-cli 用 socket+state 同步,移植成本高
|
|||
|
|
EOF
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
如果方案 A FAIL(极不可能),降级 B/C 重写 spike。
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 2 完成`:方案 A 通过,决策记录在 `SPIKE_DECISION.md`。后续 snapshot-ref.service.ts 实现该方案。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 3 [SPIKE #2]:ghost-cursor on patchright 兼容性验证
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写 spike 测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
|
|||
|
|
import { chromium } from 'patchright';
|
|||
|
|
import { createCursor } from 'ghost-cursor';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import * as fs from 'node:fs';
|
|||
|
|
|
|||
|
|
const CHROME = path.resolve(__dirname, '../../../../../../netabrowser-cli/chromium/win64/chrome.exe');
|
|||
|
|
|
|||
|
|
describe('Spike #2: ghost-cursor on patchright + neta-chromium', () => {
|
|||
|
|
jest.setTimeout(60000);
|
|||
|
|
|
|||
|
|
it('cursor.click 能在 patchright 启动的 neta-chromium 上工作', async () => {
|
|||
|
|
if (!fs.existsSync(CHROME)) {
|
|||
|
|
console.warn('SKIP: neta-chromium not found');
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
const tmpProfile = path.resolve(process.cwd(), '.spike-profile-2');
|
|||
|
|
const ctx = await chromium.launchPersistentContext(tmpProfile, {
|
|||
|
|
executablePath: CHROME,
|
|||
|
|
headless: true,
|
|||
|
|
args: ['--fingerprint=99999'],
|
|||
|
|
});
|
|||
|
|
const page = ctx.pages()[0] || (await ctx.newPage());
|
|||
|
|
|
|||
|
|
await page.goto('data:text/html,<button id=b style="position:absolute;top:200px;left:300px">Click me</button><script>document.getElementById("b").addEventListener("click",()=>document.title="CLICKED")</script>');
|
|||
|
|
|
|||
|
|
// 创建 ghost cursor
|
|||
|
|
const cursor = createCursor(page as any);
|
|||
|
|
|
|||
|
|
// 拿目标坐标
|
|||
|
|
const box = await page.locator('#b').boundingBox();
|
|||
|
|
expect(box).not.toBeNull();
|
|||
|
|
|
|||
|
|
// 用 ghost-cursor 移动并点击(贝塞尔轨迹)
|
|||
|
|
await cursor.moveTo({ x: box!.x + box!.width / 2, y: box!.y + box!.height / 2 });
|
|||
|
|
await page.mouse.down();
|
|||
|
|
await page.waitForTimeout(80);
|
|||
|
|
await page.mouse.up();
|
|||
|
|
|
|||
|
|
// 等待 click 生效
|
|||
|
|
await page.waitForFunction(() => document.title === 'CLICKED', null, { timeout: 5000 });
|
|||
|
|
expect(await page.title()).toBe('CLICKED');
|
|||
|
|
|
|||
|
|
// 验证 navigator.webdriver 仍为 false(patchright 反检测有效)
|
|||
|
|
const wd = await page.evaluate(() => navigator.webdriver);
|
|||
|
|
expect(wd).toBe(false);
|
|||
|
|
|
|||
|
|
await ctx.close();
|
|||
|
|
fs.rmSync(tmpProfile, { recursive: true, force: true });
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑 spike**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest test/modules/netaclaw/browser-daemon/spike2-ghost-cursor.test.ts
|
|||
|
|
```
|
|||
|
|
预期:PASS(含 navigator.webdriver === false)
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 写决策**
|
|||
|
|
|
|||
|
|
如果通过,append `SPIKE_DECISION.md`:
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
|
|||
|
|
# Spike #2 决策:ghost-cursor 兼容
|
|||
|
|
|
|||
|
|
✅ ghost-cursor 在 patchright + neta-chromium 上工作正常。
|
|||
|
|
- cursor.moveTo 走贝塞尔轨迹,patchright 没拦截 mouse 事件
|
|||
|
|
- click 后 navigator.webdriver 仍 false(patchright 反检测维持)
|
|||
|
|
|
|||
|
|
humanizer.service.ts 直接 `import { createCursor } from 'ghost-cursor'` 即可。
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
如果失败:选用纯自写贝塞尔(替代方案,备选脚本另准备)。
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 3 完成`:ghost-cursor 兼容性确认,可投入 humanizer 实现。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 4:chromium-launcher(路径解析 + launch args 拼装)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写 fingerprint 测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/fingerprint.service.test.ts
|
|||
|
|
import { FingerprintService } from '../../../../src/modules/netaclaw/browser-daemon/service/fingerprint.service.js';
|
|||
|
|
|
|||
|
|
describe('FingerprintService', () => {
|
|||
|
|
const svc = new FingerprintService();
|
|||
|
|
|
|||
|
|
it('fromSeed: 仅 seed 生成完整 args', () => {
|
|||
|
|
const args = svc.fromSeed(12345);
|
|||
|
|
expect(args).toEqual(expect.arrayContaining([
|
|||
|
|
'--fingerprint=12345',
|
|||
|
|
'--fingerprint-platform=Windows',
|
|||
|
|
'--fingerprint-platform-version=10.0.0',
|
|||
|
|
'--fingerprint-brand=Chrome',
|
|||
|
|
'--fingerprint-hardware-concurrency=8',
|
|||
|
|
'--fingerprint-language=zh-CN',
|
|||
|
|
]));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('merge: 指定字段覆盖默认', () => {
|
|||
|
|
const args = svc.merge({ seed: 1, language: 'en-US', hardwareConcurrency: 16 });
|
|||
|
|
expect(args).toContain('--fingerprint=1');
|
|||
|
|
expect(args).toContain('--fingerprint-language=en-US');
|
|||
|
|
expect(args).toContain('--fingerprint-hardware-concurrency=16');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('seed 不同生成不同 fingerprint arg', () => {
|
|||
|
|
expect(svc.fromSeed(1)).toContain('--fingerprint=1');
|
|||
|
|
expect(svc.fromSeed(2)).toContain('--fingerprint=2');
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend && pnpm jest fingerprint.service
|
|||
|
|
```
|
|||
|
|
预期:FAIL(service 不存在)
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现 FingerprintService**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/service/fingerprint.service.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
|
|||
|
|
|
|||
|
|
export interface FingerprintParams {
|
|||
|
|
seed?: number;
|
|||
|
|
platform?: 'Windows' | 'macOS' | 'Linux';
|
|||
|
|
platformVersion?: string;
|
|||
|
|
brand?: string;
|
|||
|
|
brandVersion?: string;
|
|||
|
|
hardwareConcurrency?: number;
|
|||
|
|
language?: string;
|
|||
|
|
timezone?: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class FingerprintService {
|
|||
|
|
/** 仅 seed → 全套默认 args */
|
|||
|
|
fromSeed(seed: number): string[] {
|
|||
|
|
return this.merge({ seed });
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
/** 自定义参数 + 默认值 → args */
|
|||
|
|
merge(p: FingerprintParams): string[] {
|
|||
|
|
const args: string[] = [];
|
|||
|
|
if (p.seed != null) args.push(`--fingerprint=${p.seed}`);
|
|||
|
|
args.push(`--fingerprint-platform=${p.platform ?? 'Windows'}`);
|
|||
|
|
args.push(`--fingerprint-platform-version=${p.platformVersion ?? '10.0.0'}`);
|
|||
|
|
args.push(`--fingerprint-brand=${p.brand ?? 'Chrome'}`);
|
|||
|
|
if (p.brandVersion) args.push(`--fingerprint-brand-version=${p.brandVersion}`);
|
|||
|
|
args.push(`--fingerprint-hardware-concurrency=${p.hardwareConcurrency ?? 8}`);
|
|||
|
|
args.push(`--fingerprint-language=${p.language ?? 'zh-CN'}`);
|
|||
|
|
if (p.timezone) args.push(`--fingerprint-timezone=${p.timezone}`);
|
|||
|
|
return args;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest fingerprint.service
|
|||
|
|
```
|
|||
|
|
预期:PASS(3 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 写 chromium-launcher 测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/chromium-launcher.test.ts
|
|||
|
|
import { resolveChromiumPath, buildLaunchArgs } from '../../../../src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.js';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
|
|||
|
|
describe('chromium-launcher', () => {
|
|||
|
|
describe('resolveChromiumPath', () => {
|
|||
|
|
afterEach(() => { delete process.env.NETA_CHROMIUM_PATH; });
|
|||
|
|
|
|||
|
|
it('优先环境变量', () => {
|
|||
|
|
process.env.NETA_CHROMIUM_PATH = '/custom/chrome.exe';
|
|||
|
|
expect(resolveChromiumPath({ isPkg: false })).toBe('/custom/chrome.exe');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('pkg 模式:execPath 同目录 chromium/win64/chrome.exe', () => {
|
|||
|
|
const result = resolveChromiumPath({ isPkg: true, execPath: 'C:/Program Files/Neta/backend.exe' });
|
|||
|
|
expect(result).toMatch(/Program Files\/Neta\/chromium\/win64\/chrome\.exe$/);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('dev 模式:monorepo packages/netabrowser-cli/chromium/win64/chrome.exe', () => {
|
|||
|
|
const result = resolveChromiumPath({ isPkg: false });
|
|||
|
|
expect(result).toMatch(/netabrowser-cli\/chromium\/win64\/chrome\.exe$/);
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
describe('buildLaunchArgs', () => {
|
|||
|
|
it('合并指纹 args + 用户自定义 args', () => {
|
|||
|
|
const args = buildLaunchArgs({
|
|||
|
|
fingerprintArgs: ['--fingerprint=42', '--fingerprint-language=zh-CN'],
|
|||
|
|
extraArgs: ['--disable-popup-blocking'],
|
|||
|
|
});
|
|||
|
|
expect(args).toEqual(expect.arrayContaining([
|
|||
|
|
'--fingerprint=42',
|
|||
|
|
'--fingerprint-language=zh-CN',
|
|||
|
|
'--disable-popup-blocking',
|
|||
|
|
]));
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: 实现 chromium-launcher**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/chromium-launcher.ts
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
|
|||
|
|
export interface ResolveOpts {
|
|||
|
|
isPkg?: boolean;
|
|||
|
|
execPath?: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
/** 解析 neta-chromium 可执行文件路径 */
|
|||
|
|
export function resolveChromiumPath(opts: ResolveOpts = {}): string {
|
|||
|
|
if (process.env.NETA_CHROMIUM_PATH) return process.env.NETA_CHROMIUM_PATH;
|
|||
|
|
|
|||
|
|
const isPkg = opts.isPkg ?? !!(process as any).pkg;
|
|||
|
|
if (isPkg) {
|
|||
|
|
const execPath = opts.execPath ?? process.execPath;
|
|||
|
|
return path.join(path.dirname(execPath), 'chromium', 'win64', 'chrome.exe').replace(/\\/g, '/');
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
// dev 模式:从 backend src 出发,定位 monorepo 内 packages/netabrowser-cli/chromium/win64/chrome.exe
|
|||
|
|
// 编译后 __dirname = packages/backend/dist/modules/netaclaw/browser-daemon/runtime
|
|||
|
|
// 路径计算:runtime → browser-daemon → netaclaw → modules → dist → backend → packages → root(7 层 ..)
|
|||
|
|
// 然后 root/packages/netabrowser-cli/chromium/win64/chrome.exe
|
|||
|
|
return path.resolve(__dirname, '../../../../../../../packages/netabrowser-cli/chromium/win64/chrome.exe').replace(/\\/g, '/');
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface LaunchArgsOpts {
|
|||
|
|
fingerprintArgs: string[];
|
|||
|
|
extraArgs?: string[];
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export function buildLaunchArgs(opts: LaunchArgsOpts): string[] {
|
|||
|
|
return [...opts.fingerprintArgs, ...(opts.extraArgs ?? [])];
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 7: 跑 chromium-launcher 测试**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest chromium-launcher
|
|||
|
|
```
|
|||
|
|
预期:PASS(4 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 8: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 4 完成`:路径解析 + 指纹参数生成 + launch args 拼装可用。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 4b:BrowserDataDir 独立解析(**新增 P0 修复**)
|
|||
|
|
|
|||
|
|
**问题**:`comm/data-dir.ts:resolveDataDir()` 在 dev 模式 fallback 到 `<cwd>/dist`。把 `browser-profiles/` `states/` 放 dist 下,每次 `pnpm build` 会清空,状态全丢。
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/browser-data-dir.test.ts
|
|||
|
|
import { resolveBrowserDataDir, getProfileDir, getStateDir } from '../../../../src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.js';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
|
|||
|
|
describe('BrowserDataDir', () => {
|
|||
|
|
afterEach(() => {
|
|||
|
|
delete process.env.NETA_BROWSER_DATA_DIR;
|
|||
|
|
delete process.env.NETA_DATA_DIR;
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('优先 NETA_BROWSER_DATA_DIR 环境变量', () => {
|
|||
|
|
process.env.NETA_BROWSER_DATA_DIR = '/custom/browser';
|
|||
|
|
expect(resolveBrowserDataDir({ isPkg: false })).toBe('/custom/browser');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('pkg 模式:dataDir/.netabrowser', () => {
|
|||
|
|
const r = resolveBrowserDataDir({ isPkg: true, execDir: 'C:/Program Files/Neta' });
|
|||
|
|
expect(r.replace(/\\/g, '/')).toBe('C:/Program Files/Neta/data/.netabrowser');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('dev 模式:monorepo/.netabrowser-data(不复用 dist)', () => {
|
|||
|
|
const r = resolveBrowserDataDir({ isPkg: false, monorepoRoot: '/repo' });
|
|||
|
|
expect(r.replace(/\\/g, '/')).toBe('/repo/.netabrowser-data');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('getProfileDir / getStateDir 返回子路径', () => {
|
|||
|
|
process.env.NETA_BROWSER_DATA_DIR = '/x';
|
|||
|
|
expect(getProfileDir('s1').replace(/\\/g, '/')).toBe('/x/profiles/s1');
|
|||
|
|
expect(getStateDir('s1').replace(/\\/g, '/')).toBe('/x/states/s1.json');
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 实现**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/browser-data-dir.ts
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import { resolveDataDir } from '../../../../comm/data-dir.js';
|
|||
|
|
|
|||
|
|
export interface ResolveBrowserDataDirOpts {
|
|||
|
|
isPkg?: boolean;
|
|||
|
|
execDir?: string;
|
|||
|
|
monorepoRoot?: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
/**
|
|||
|
|
* netabrowser-cli 自己的状态根目录,独立于 backend dist/。
|
|||
|
|
* - prod: <dataDir>/.netabrowser/
|
|||
|
|
* - dev: <monorepo-root>/.netabrowser-data/ (不放 dist 下,防 build 清空)
|
|||
|
|
*/
|
|||
|
|
export function resolveBrowserDataDir(opts: ResolveBrowserDataDirOpts = {}): string {
|
|||
|
|
if (process.env.NETA_BROWSER_DATA_DIR) return path.resolve(process.env.NETA_BROWSER_DATA_DIR);
|
|||
|
|
const isPkg = opts.isPkg ?? !!(process as any).pkg;
|
|||
|
|
if (isPkg) {
|
|||
|
|
const dataDir = resolveDataDir({ isPkg: true, execDir: opts.execDir });
|
|||
|
|
return path.join(dataDir, '.netabrowser');
|
|||
|
|
}
|
|||
|
|
// dev: 从 backend 出发上溯到 monorepo root
|
|||
|
|
const monorepoRoot = opts.monorepoRoot
|
|||
|
|
?? path.resolve(__dirname, '../../../../../../../');
|
|||
|
|
return path.join(monorepoRoot, '.netabrowser-data');
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export function getProfileDir(sessionName: string): string {
|
|||
|
|
return path.join(resolveBrowserDataDir(), 'profiles', sessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export function getStateDir(sessionName: string): string {
|
|||
|
|
return path.join(resolveBrowserDataDir(), 'states', `${sessionName}.json`);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest browser-data-dir
|
|||
|
|
```
|
|||
|
|
预期:PASS(4 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 修改 Task 7 daemon.service 中的 import**
|
|||
|
|
|
|||
|
|
将 daemon.service.ts 的:
|
|||
|
|
```typescript
|
|||
|
|
import { resolveDataDir } from '../../../../comm/data-dir.js';
|
|||
|
|
// ...
|
|||
|
|
const profileRoot = path.join(resolveDataDir(), 'browser-profiles');
|
|||
|
|
const profilePath = path.join(profileRoot, opts.profileDir ?? opts.sessionName);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
改为:
|
|||
|
|
```typescript
|
|||
|
|
import { getProfileDir } from '../runtime/browser-data-dir.js';
|
|||
|
|
// ...
|
|||
|
|
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
同样将 saveState/loadState 用到的状态文件路径改为 `getStateDir(sessionName)`。
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 更新 .gitignore**
|
|||
|
|
|
|||
|
|
把 `.netabrowser-data/` 加入 monorepo 根 `.gitignore`:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
echo ".netabrowser-data/" >> C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo/.gitignore
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 4b 完成`:BrowserDataDir 独立,不复用 dist,build 不会清空状态。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 5:SessionRegistry(含串行化锁)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/session-registry.test.ts
|
|||
|
|
import { SessionRegistry } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js';
|
|||
|
|
|
|||
|
|
describe('SessionRegistry', () => {
|
|||
|
|
let registry: SessionRegistry;
|
|||
|
|
beforeEach(() => { registry = new SessionRegistry(); });
|
|||
|
|
|
|||
|
|
it('register/get/has/list 基础 CRUD', () => {
|
|||
|
|
const fakeCtx = { id: 'A' } as any;
|
|||
|
|
registry.register('s1', fakeCtx);
|
|||
|
|
expect(registry.has('s1')).toBe(true);
|
|||
|
|
expect(registry.get('s1')).toBe(fakeCtx);
|
|||
|
|
expect(registry.list()).toEqual([{ sessionName: 's1', context: fakeCtx, lastUsedAt: expect.any(Date) }]);
|
|||
|
|
registry.unregister('s1');
|
|||
|
|
expect(registry.has('s1')).toBe(false);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('touch 更新 lastUsedAt', async () => {
|
|||
|
|
registry.register('s1', { id: 'A' } as any);
|
|||
|
|
const before = registry.list()[0].lastUsedAt;
|
|||
|
|
await new Promise(r => setTimeout(r, 10));
|
|||
|
|
registry.touch('s1');
|
|||
|
|
const after = registry.list()[0].lastUsedAt;
|
|||
|
|
expect(after.getTime()).toBeGreaterThan(before.getTime());
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('withLock: 同 sessionName 串行化', async () => {
|
|||
|
|
const order: string[] = [];
|
|||
|
|
const job = (id: string, ms: number) => async () => {
|
|||
|
|
order.push(`start-${id}`);
|
|||
|
|
await new Promise(r => setTimeout(r, ms));
|
|||
|
|
order.push(`end-${id}`);
|
|||
|
|
};
|
|||
|
|
await Promise.all([
|
|||
|
|
registry.withLock('s1', job('A', 30)),
|
|||
|
|
registry.withLock('s1', job('B', 5)),
|
|||
|
|
registry.withLock('s1', job('C', 5)),
|
|||
|
|
]);
|
|||
|
|
expect(order).toEqual(['start-A', 'end-A', 'start-B', 'end-B', 'start-C', 'end-C']);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('withLock: 不同 sessionName 并发', async () => {
|
|||
|
|
const order: string[] = [];
|
|||
|
|
await Promise.all([
|
|||
|
|
registry.withLock('s1', async () => { order.push('s1-start'); await new Promise(r => setTimeout(r, 30)); order.push('s1-end'); }),
|
|||
|
|
registry.withLock('s2', async () => { order.push('s2-start'); await new Promise(r => setTimeout(r, 10)); order.push('s2-end'); }),
|
|||
|
|
]);
|
|||
|
|
// s2 应该在 s1 完成前先 end
|
|||
|
|
expect(order.indexOf('s2-end')).toBeLessThan(order.indexOf('s1-end'));
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest session-registry
|
|||
|
|
```
|
|||
|
|
预期:FAIL
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-registry.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
|
|||
|
|
|
|||
|
|
export interface SessionEntry {
|
|||
|
|
sessionName: string;
|
|||
|
|
context: any; // patchright BrowserContext
|
|||
|
|
lastUsedAt: Date;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class SessionRegistry {
|
|||
|
|
private entries = new Map<string, SessionEntry>();
|
|||
|
|
private locks = new Map<string, Promise<void>>();
|
|||
|
|
|
|||
|
|
register(sessionName: string, context: any): void {
|
|||
|
|
this.entries.set(sessionName, { sessionName, context, lastUsedAt: new Date() });
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
unregister(sessionName: string): void {
|
|||
|
|
this.entries.delete(sessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
has(sessionName: string): boolean {
|
|||
|
|
return this.entries.has(sessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
get(sessionName: string): any | undefined {
|
|||
|
|
return this.entries.get(sessionName)?.context;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
touch(sessionName: string): void {
|
|||
|
|
const e = this.entries.get(sessionName);
|
|||
|
|
if (e) e.lastUsedAt = new Date();
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
list(): SessionEntry[] {
|
|||
|
|
return [...this.entries.values()];
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
/** 同 sessionName 操作严格串行化 */
|
|||
|
|
async withLock<T>(sessionName: string, fn: () => Promise<T>): Promise<T> {
|
|||
|
|
const prev = this.locks.get(sessionName) ?? Promise.resolve();
|
|||
|
|
let release!: () => void;
|
|||
|
|
const next = new Promise<void>(r => (release = r));
|
|||
|
|
this.locks.set(sessionName, prev.then(() => next));
|
|||
|
|
try {
|
|||
|
|
await prev;
|
|||
|
|
return await fn();
|
|||
|
|
} finally {
|
|||
|
|
release();
|
|||
|
|
if (this.locks.get(sessionName) === next) this.locks.delete(sessionName);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest session-registry
|
|||
|
|
```
|
|||
|
|
预期:PASS(4 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 5 完成`:SessionRegistry 含锁机制,并发安全。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 6:SessionScheduler(LRU 软上限 + 队列)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/session-scheduler.test.ts
|
|||
|
|
import { SessionScheduler } from '../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js';
|
|||
|
|
|
|||
|
|
describe('SessionScheduler', () => {
|
|||
|
|
it('未触达上限直接通过', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 5 });
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
const slot = await sch.acquireSlot('s2', 'normal', false);
|
|||
|
|
expect(slot.granted).toBe(true);
|
|||
|
|
expect(slot.evictedSessionName).toBeUndefined();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('触达上限:LRU 回收最久未用的 idle session', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 2, idleTimeoutMs: 1000 });
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
await new Promise(r => setTimeout(r, 5));
|
|||
|
|
sch.recordActivate('s2');
|
|||
|
|
sch.markIdle('s1');
|
|||
|
|
sch.markIdle('s2');
|
|||
|
|
// s1 比 s2 先 idle,所以 s1 是 LRU
|
|||
|
|
const slot = await sch.acquireSlot('s3', 'normal', false);
|
|||
|
|
expect(slot.granted).toBe(true);
|
|||
|
|
expect(slot.evictedSessionName).toBe('s1');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('全部 active 无 idle 可回收:fail-fast 503', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 1 });
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
const slot = await sch.acquireSlot('s2', 'normal', false);
|
|||
|
|
expect(slot.granted).toBe(false);
|
|||
|
|
expect(slot.reason).toBe('NO_IDLE_SESSION_TO_EVICT');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('high 优先级抢占 low 优先级', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 1 });
|
|||
|
|
sch.recordActivate('s1', 'low');
|
|||
|
|
const slot = await sch.acquireSlot('s2', 'high', false);
|
|||
|
|
expect(slot.granted).toBe(true);
|
|||
|
|
expect(slot.evictedSessionName).toBe('s1');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('queue=true:等待空位(最长 30s)', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 1, queueTimeoutMs: 200 });
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
const p = sch.acquireSlot('s2', 'normal', true);
|
|||
|
|
setTimeout(() => sch.markIdle('s1'), 50);
|
|||
|
|
setTimeout(() => sch.recordDeactivate('s1'), 60);
|
|||
|
|
const slot = await p;
|
|||
|
|
expect(slot.granted).toBe(true);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('getStats 返回 active/idle/total', () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 5 });
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
sch.recordActivate('s2');
|
|||
|
|
sch.markIdle('s1');
|
|||
|
|
expect(sch.getStats()).toEqual({
|
|||
|
|
activeCount: 1,
|
|||
|
|
idleCount: 1,
|
|||
|
|
totalCount: 2,
|
|||
|
|
maxActiveSessions: 5,
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest session-scheduler
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
|
|||
|
|
|
|||
|
|
export type Priority = 'low' | 'normal' | 'high';
|
|||
|
|
const PRIORITY_RANK: Record<Priority, number> = { low: 0, normal: 1, high: 2 };
|
|||
|
|
|
|||
|
|
interface SessionState {
|
|||
|
|
sessionName: string;
|
|||
|
|
priority: Priority;
|
|||
|
|
status: 'active' | 'idle';
|
|||
|
|
activatedAt: Date;
|
|||
|
|
lastUsedAt: Date;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface SchedulerOptions {
|
|||
|
|
maxActiveSessions?: number;
|
|||
|
|
idleTimeoutMs?: number;
|
|||
|
|
queueTimeoutMs?: number;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface AcquireResult {
|
|||
|
|
granted: boolean;
|
|||
|
|
reason?: string;
|
|||
|
|
evictedSessionName?: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class SessionScheduler {
|
|||
|
|
private sessions = new Map<string, SessionState>();
|
|||
|
|
private waiters: Array<{ resolve: (v: AcquireResult) => void; sessionName: string; priority: Priority; expireAt: number }> = [];
|
|||
|
|
private maxActiveSessions: number;
|
|||
|
|
private queueTimeoutMs: number;
|
|||
|
|
|
|||
|
|
constructor(opts: SchedulerOptions = {}) {
|
|||
|
|
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
|
|||
|
|
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
recordActivate(sessionName: string, priority: Priority = 'normal'): void {
|
|||
|
|
const now = new Date();
|
|||
|
|
this.sessions.set(sessionName, { sessionName, priority, status: 'active', activatedAt: now, lastUsedAt: now });
|
|||
|
|
this.tryProcessWaiters();
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
markIdle(sessionName: string): void {
|
|||
|
|
const s = this.sessions.get(sessionName);
|
|||
|
|
if (s) s.status = 'idle';
|
|||
|
|
this.tryProcessWaiters();
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
recordDeactivate(sessionName: string): void {
|
|||
|
|
this.sessions.delete(sessionName);
|
|||
|
|
this.tryProcessWaiters();
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
touch(sessionName: string): void {
|
|||
|
|
const s = this.sessions.get(sessionName);
|
|||
|
|
if (s) {
|
|||
|
|
s.status = 'active';
|
|||
|
|
s.lastUsedAt = new Date();
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async acquireSlot(sessionName: string, priority: Priority, queue: boolean): Promise<AcquireResult> {
|
|||
|
|
const direct = this.tryAcquireImmediate(sessionName, priority);
|
|||
|
|
if (direct.granted || !queue) return direct;
|
|||
|
|
if (direct.reason !== 'NO_IDLE_SESSION_TO_EVICT' && direct.reason !== 'AT_CAPACITY') {
|
|||
|
|
return direct;
|
|||
|
|
}
|
|||
|
|
// 排队
|
|||
|
|
return new Promise<AcquireResult>(resolve => {
|
|||
|
|
const expireAt = Date.now() + this.queueTimeoutMs;
|
|||
|
|
const waiter = { resolve, sessionName, priority, expireAt };
|
|||
|
|
this.waiters.push(waiter);
|
|||
|
|
const timer = setTimeout(() => {
|
|||
|
|
const i = this.waiters.indexOf(waiter);
|
|||
|
|
if (i >= 0) {
|
|||
|
|
this.waiters.splice(i, 1);
|
|||
|
|
resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
|
|||
|
|
}
|
|||
|
|
}, this.queueTimeoutMs);
|
|||
|
|
// resolve 时清除 timer
|
|||
|
|
const origResolve = waiter.resolve;
|
|||
|
|
waiter.resolve = (v) => { clearTimeout(timer); origResolve(v); };
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private tryAcquireImmediate(sessionName: string, priority: Priority): AcquireResult {
|
|||
|
|
if (this.sessions.has(sessionName)) {
|
|||
|
|
// 已存在:activate 并返回成功
|
|||
|
|
this.touch(sessionName);
|
|||
|
|
return { granted: true };
|
|||
|
|
}
|
|||
|
|
const activeCount = [...this.sessions.values()].filter(s => s.status === 'active').length;
|
|||
|
|
if (activeCount < this.maxActiveSessions) {
|
|||
|
|
return { granted: true };
|
|||
|
|
}
|
|||
|
|
// 触达上限:尝试 LRU 回收 idle
|
|||
|
|
const idleSessions = [...this.sessions.values()]
|
|||
|
|
.filter(s => s.status === 'idle')
|
|||
|
|
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
|
|||
|
|
if (idleSessions.length > 0) {
|
|||
|
|
const evict = idleSessions[0];
|
|||
|
|
this.sessions.delete(evict.sessionName);
|
|||
|
|
return { granted: true, evictedSessionName: evict.sessionName };
|
|||
|
|
}
|
|||
|
|
// 无 idle,尝试优先级抢占
|
|||
|
|
const lowerPriority = [...this.sessions.values()]
|
|||
|
|
.filter(s => PRIORITY_RANK[s.priority] < PRIORITY_RANK[priority])
|
|||
|
|
.sort((a, b) => a.lastUsedAt.getTime() - b.lastUsedAt.getTime());
|
|||
|
|
if (lowerPriority.length > 0) {
|
|||
|
|
const evict = lowerPriority[0];
|
|||
|
|
this.sessions.delete(evict.sessionName);
|
|||
|
|
return { granted: true, evictedSessionName: evict.sessionName };
|
|||
|
|
}
|
|||
|
|
return { granted: false, reason: 'NO_IDLE_SESSION_TO_EVICT' };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private tryProcessWaiters(): void {
|
|||
|
|
while (this.waiters.length > 0) {
|
|||
|
|
const w = this.waiters[0];
|
|||
|
|
if (Date.now() > w.expireAt) {
|
|||
|
|
this.waiters.shift();
|
|||
|
|
w.resolve({ granted: false, reason: 'QUEUE_TIMEOUT' });
|
|||
|
|
continue;
|
|||
|
|
}
|
|||
|
|
const r = this.tryAcquireImmediate(w.sessionName, w.priority);
|
|||
|
|
if (r.granted) {
|
|||
|
|
this.waiters.shift();
|
|||
|
|
w.resolve(r);
|
|||
|
|
} else {
|
|||
|
|
break;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
getStats(): { activeCount: number; idleCount: number; totalCount: number; maxActiveSessions: number } {
|
|||
|
|
const all = [...this.sessions.values()];
|
|||
|
|
return {
|
|||
|
|
activeCount: all.filter(s => s.status === 'active').length,
|
|||
|
|
idleCount: all.filter(s => s.status === 'idle').length,
|
|||
|
|
totalCount: all.length,
|
|||
|
|
maxActiveSessions: this.maxActiveSessions,
|
|||
|
|
};
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest session-scheduler
|
|||
|
|
```
|
|||
|
|
预期:PASS(6 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 6 完成`:调度器 LRU+优先级+队列+stats 完整。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 7:BrowserDaemonService.open/close 基础生命周期
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试(mock patchright + dependencies)**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/daemon.service.test.ts
|
|||
|
|
import { BrowserDaemonService } from '../../../../src/modules/netaclaw/browser-daemon/service/daemon.service.js';
|
|||
|
|
|
|||
|
|
describe('BrowserDaemonService.open/close', () => {
|
|||
|
|
let svc: BrowserDaemonService;
|
|||
|
|
let mockChromium: any;
|
|||
|
|
let mockContext: any;
|
|||
|
|
let mockPage: any;
|
|||
|
|
|
|||
|
|
beforeEach(() => {
|
|||
|
|
mockPage = { goto: jest.fn(), close: jest.fn(), url: () => 'about:blank' };
|
|||
|
|
mockContext = {
|
|||
|
|
pages: () => [mockPage],
|
|||
|
|
newPage: jest.fn().mockResolvedValue(mockPage),
|
|||
|
|
close: jest.fn().mockResolvedValue(undefined),
|
|||
|
|
storageState: jest.fn().mockResolvedValue({ cookies: [], origins: [] }),
|
|||
|
|
};
|
|||
|
|
mockChromium = { launchPersistentContext: jest.fn().mockResolvedValue(mockContext) };
|
|||
|
|
|
|||
|
|
svc = new BrowserDaemonService();
|
|||
|
|
(svc as any).registry = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js').SessionRegistry)();
|
|||
|
|
(svc as any).scheduler = new (require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js').SessionScheduler)({ maxActiveSessions: 5 });
|
|||
|
|
(svc as any).fingerprint = { fromSeed: (s: number) => [`--fingerprint=${s}`] };
|
|||
|
|
(svc as any).chromium = mockChromium;
|
|||
|
|
(svc as any).resolveChromiumPath = () => '/fake/chrome.exe';
|
|||
|
|
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('open: 启动 chromium + 注册 session', async () => {
|
|||
|
|
const r = await svc.open({ sessionName: 's1', url: 'https://example.com', fingerprintSeed: 42 });
|
|||
|
|
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
|
|||
|
|
expect.any(String),
|
|||
|
|
expect.objectContaining({
|
|||
|
|
executablePath: '/fake/chrome.exe',
|
|||
|
|
args: expect.arrayContaining(['--fingerprint=42']),
|
|||
|
|
})
|
|||
|
|
);
|
|||
|
|
expect(r.sessionName).toBe('s1');
|
|||
|
|
expect((svc as any).registry.has('s1')).toBe(true);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('open: sessionName 已存在 → 409', async () => {
|
|||
|
|
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
|
|||
|
|
await expect(svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 }))
|
|||
|
|
.rejects.toThrow(/already exists|conflict/i);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('close: 关闭 context + 注销 session', async () => {
|
|||
|
|
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
|
|||
|
|
await svc.close('s1');
|
|||
|
|
expect(mockContext.close).toHaveBeenCalled();
|
|||
|
|
expect((svc as any).registry.has('s1')).toBe(false);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('list: 返回所有 session 信息', async () => {
|
|||
|
|
await svc.open({ sessionName: 's1', url: 'https://e.com', fingerprintSeed: 1 });
|
|||
|
|
const list = svc.list();
|
|||
|
|
expect(list).toEqual([expect.objectContaining({ sessionName: 's1' })]);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('proxy 配置传给 patchright', async () => {
|
|||
|
|
await svc.open({
|
|||
|
|
sessionName: 's1',
|
|||
|
|
url: 'https://e.com',
|
|||
|
|
fingerprintSeed: 1,
|
|||
|
|
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
|
|||
|
|
});
|
|||
|
|
expect(mockChromium.launchPersistentContext).toHaveBeenCalledWith(
|
|||
|
|
expect.any(String),
|
|||
|
|
expect.objectContaining({
|
|||
|
|
proxy: { server: 'http://1.2.3.4:8080', username: 'u', password: 'p' },
|
|||
|
|
})
|
|||
|
|
);
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest daemon.service
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现 daemon.service.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum, Inject, Logger, ILogger } from '@midwayjs/core';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import * as fs from 'node:fs';
|
|||
|
|
import { chromium } from 'patchright';
|
|||
|
|
import { SessionRegistry } from '../runtime/session-registry.js';
|
|||
|
|
import { SessionScheduler, Priority } from '../runtime/session-scheduler.js';
|
|||
|
|
import { FingerprintService } from './fingerprint.service.js';
|
|||
|
|
import { resolveChromiumPath, buildLaunchArgs } from '../runtime/chromium-launcher.js';
|
|||
|
|
import { getProfileDir } from '../runtime/browser-data-dir.js';
|
|||
|
|
|
|||
|
|
export type HumanizeMode = 'full' | 'fast' | 'off';
|
|||
|
|
|
|||
|
|
export interface OpenOpts {
|
|||
|
|
sessionName: string;
|
|||
|
|
url: string;
|
|||
|
|
fingerprintSeed?: number;
|
|||
|
|
proxy?: { server: string; username?: string; password?: string };
|
|||
|
|
profileDir?: string;
|
|||
|
|
headed?: boolean;
|
|||
|
|
priority?: Priority;
|
|||
|
|
queue?: boolean;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface SessionInfo {
|
|||
|
|
sessionName: string;
|
|||
|
|
url: string;
|
|||
|
|
pageCount: number;
|
|||
|
|
status: 'active' | 'idle';
|
|||
|
|
lastUsedAt: Date;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class BrowserDaemonService {
|
|||
|
|
@Inject() registry: SessionRegistry;
|
|||
|
|
@Inject() scheduler: SessionScheduler;
|
|||
|
|
@Inject() fingerprint: FingerprintService;
|
|||
|
|
@Logger() logger: ILogger;
|
|||
|
|
|
|||
|
|
// 注入点便于测试覆盖
|
|||
|
|
protected chromium: any = chromium;
|
|||
|
|
protected resolveChromiumPath = resolveChromiumPath;
|
|||
|
|
|
|||
|
|
/**
|
|||
|
|
* 注:v2 修复 P1-8 后 service 不再维护 modeMap(保持 stateless)。
|
|||
|
|
* humanize-mode 完全由调用方每次传:
|
|||
|
|
* - CLI 端:`--mode` 命令级 > `NETA_BROWSER_HUMANIZE_MODE` 环境变量 > 默认 'full'
|
|||
|
|
* - HTTP/Service 调用方:每次方法调用通过参数传入
|
|||
|
|
*/
|
|||
|
|
|
|||
|
|
async open(opts: OpenOpts): Promise<{ sessionName: string; url: string; pageCount: number }> {
|
|||
|
|
return this.registry.withLock(opts.sessionName, async () => {
|
|||
|
|
if (this.registry.has(opts.sessionName)) {
|
|||
|
|
const err: any = new Error(`Session '${opts.sessionName}' already exists`);
|
|||
|
|
err.statusCode = 409;
|
|||
|
|
throw err;
|
|||
|
|
}
|
|||
|
|
const slot = await this.scheduler.acquireSlot(
|
|||
|
|
opts.sessionName,
|
|||
|
|
opts.priority ?? 'normal',
|
|||
|
|
opts.queue ?? false,
|
|||
|
|
);
|
|||
|
|
if (!slot.granted) {
|
|||
|
|
const err: any = new Error(`No slot available: ${slot.reason}`);
|
|||
|
|
err.statusCode = 503;
|
|||
|
|
err.retryAfter = 30;
|
|||
|
|
throw err;
|
|||
|
|
}
|
|||
|
|
if (slot.evictedSessionName) {
|
|||
|
|
await this.closeInternal(slot.evictedSessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
const profilePath = getProfileDir(opts.profileDir ?? opts.sessionName);
|
|||
|
|
fs.mkdirSync(profilePath, { recursive: true });
|
|||
|
|
|
|||
|
|
const fingerprintArgs = this.fingerprint.fromSeed(opts.fingerprintSeed ?? Math.floor(Math.random() * 100000));
|
|||
|
|
const args = buildLaunchArgs({ fingerprintArgs });
|
|||
|
|
|
|||
|
|
this.logger.info(`[browser-daemon] launching ${opts.sessionName} → ${opts.url}`);
|
|||
|
|
const ctx = await this.chromium.launchPersistentContext(profilePath, {
|
|||
|
|
executablePath: this.resolveChromiumPath(),
|
|||
|
|
headless: !opts.headed,
|
|||
|
|
args,
|
|||
|
|
proxy: opts.proxy,
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
const page = ctx.pages()[0] ?? (await ctx.newPage());
|
|||
|
|
await page.goto(opts.url);
|
|||
|
|
|
|||
|
|
this.registry.register(opts.sessionName, ctx);
|
|||
|
|
this.scheduler.recordActivate(opts.sessionName, opts.priority ?? 'normal');
|
|||
|
|
|
|||
|
|
return { sessionName: opts.sessionName, url: page.url(), pageCount: ctx.pages().length };
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async close(sessionName: string): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, () => this.closeInternal(sessionName));
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private async closeInternal(sessionName: string): Promise<void> {
|
|||
|
|
const ctx = this.registry.get(sessionName);
|
|||
|
|
if (!ctx) return;
|
|||
|
|
try {
|
|||
|
|
await ctx.close();
|
|||
|
|
} catch (e: any) {
|
|||
|
|
this.logger.warn(`[browser-daemon] close ${sessionName} failed: ${e.message}`);
|
|||
|
|
}
|
|||
|
|
this.registry.unregister(sessionName);
|
|||
|
|
this.scheduler.recordDeactivate(sessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
list(): SessionInfo[] {
|
|||
|
|
return this.registry.list().map(e => {
|
|||
|
|
const ctx = e.context;
|
|||
|
|
const page = ctx.pages?.()[0];
|
|||
|
|
return {
|
|||
|
|
sessionName: e.sessionName,
|
|||
|
|
url: page?.url?.() ?? '',
|
|||
|
|
pageCount: ctx.pages?.().length ?? 0,
|
|||
|
|
status: 'active',
|
|||
|
|
lastUsedAt: e.lastUsedAt,
|
|||
|
|
};
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
getStats() {
|
|||
|
|
return this.scheduler.getStats();
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
getPageOrFail(sessionName: string) {
|
|||
|
|
const ctx = this.registry.get(sessionName);
|
|||
|
|
if (!ctx) throw new Error(`Session '${sessionName}' not found`);
|
|||
|
|
const page = ctx.pages()[0];
|
|||
|
|
if (!page) throw new Error(`Session '${sessionName}' has no page`);
|
|||
|
|
this.registry.touch(sessionName);
|
|||
|
|
this.scheduler.touch(sessionName);
|
|||
|
|
return { ctx, page };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest daemon.service
|
|||
|
|
```
|
|||
|
|
预期:PASS(5 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 7 完成`:daemon.service open/close/list 可用,含锁、调度、指纹、launch。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 8:HumanizerService(三档拟人化)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/humanizer.service.test.ts
|
|||
|
|
import { HumanizerService } from '../../../../src/modules/netaclaw/browser-daemon/service/humanizer.service.js';
|
|||
|
|
|
|||
|
|
describe('HumanizerService', () => {
|
|||
|
|
let svc: HumanizerService;
|
|||
|
|
let mockPage: any;
|
|||
|
|
let mockMouse: any;
|
|||
|
|
let mockCursor: any;
|
|||
|
|
|
|||
|
|
beforeEach(() => {
|
|||
|
|
mockMouse = { down: jest.fn(), up: jest.fn(), move: jest.fn(), wheel: jest.fn() };
|
|||
|
|
mockPage = { mouse: mockMouse, waitForTimeout: jest.fn() };
|
|||
|
|
mockCursor = { moveTo: jest.fn().mockResolvedValue(undefined) };
|
|||
|
|
svc = new HumanizerService();
|
|||
|
|
(svc as any).createCursor = () => mockCursor;
|
|||
|
|
(svc as any).rand = (a: number, b: number) => (a + b) / 2; // deterministic
|
|||
|
|
(svc as any).chance = (_: number) => false; // never trigger random branches
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('full 模式: ghost-cursor 移动 + 视觉停顿 + mousedown/up', async () => {
|
|||
|
|
await svc.click(mockPage, { x: 100, y: 200 }, 'full');
|
|||
|
|
expect(mockCursor.moveTo).toHaveBeenCalledWith({ x: 100, y: 200 }, expect.any(Object));
|
|||
|
|
expect(mockMouse.down).toHaveBeenCalled();
|
|||
|
|
expect(mockMouse.up).toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('fast 模式: 不走 ghost-cursor,直接 mouse.move + 短延迟', async () => {
|
|||
|
|
await svc.click(mockPage, { x: 100, y: 200 }, 'fast');
|
|||
|
|
expect(mockCursor.moveTo).not.toHaveBeenCalled();
|
|||
|
|
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
|
|||
|
|
expect(mockMouse.down).toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('off 模式: 立即 click,无延迟', async () => {
|
|||
|
|
(svc as any).rand = jest.fn();
|
|||
|
|
await svc.click(mockPage, { x: 100, y: 200 }, 'off');
|
|||
|
|
expect(mockCursor.moveTo).not.toHaveBeenCalled();
|
|||
|
|
expect(mockMouse.move).toHaveBeenCalledWith(100, 200);
|
|||
|
|
expect(mockMouse.down).toHaveBeenCalled();
|
|||
|
|
expect((svc as any).rand).not.toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('type full: 字符间随机间隔,5% 概率错字(mock 不触发)', async () => {
|
|||
|
|
const keyboard = { type: jest.fn(), press: jest.fn() };
|
|||
|
|
mockPage.keyboard = keyboard;
|
|||
|
|
await svc.type(mockPage, 'hello', 'full');
|
|||
|
|
// 'hello' 5 个字符 → keyboard.type 应该被调 5 次
|
|||
|
|
expect(keyboard.type).toHaveBeenCalledTimes(5);
|
|||
|
|
expect(keyboard.type).toHaveBeenNthCalledWith(1, 'h');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('type off: 直接整体输入', async () => {
|
|||
|
|
const keyboard = { type: jest.fn() };
|
|||
|
|
mockPage.keyboard = keyboard;
|
|||
|
|
await svc.type(mockPage, 'hello', 'off');
|
|||
|
|
expect(keyboard.type).toHaveBeenCalledWith('hello', undefined);
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest humanizer.service
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现 humanizer.service.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/service/humanizer.service.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
|
|||
|
|
import { createCursor } from 'ghost-cursor';
|
|||
|
|
|
|||
|
|
export type HumanizeMode = 'full' | 'fast' | 'off';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class HumanizerService {
|
|||
|
|
// 注入点便于测试
|
|||
|
|
protected createCursor = createCursor;
|
|||
|
|
protected rand(min: number, max: number): number {
|
|||
|
|
return min + Math.floor(Math.random() * (max - min));
|
|||
|
|
}
|
|||
|
|
protected chance(p: number): boolean {
|
|||
|
|
return Math.random() < p;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async click(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
|
|||
|
|
if (mode === 'full') {
|
|||
|
|
// ghost-cursor 是 puppeteer 类型,patchright Page 与之不兼容,需要 as any cast
|
|||
|
|
const cursor = this.createCursor(page as any);
|
|||
|
|
await cursor.moveTo(point, { randomizeMoveDelay: true, moveDelay: this.rand(50, 150) });
|
|||
|
|
await this.sleep(this.rand(100, 300));
|
|||
|
|
await page.mouse.down();
|
|||
|
|
await this.sleep(this.rand(50, 150));
|
|||
|
|
await page.mouse.up();
|
|||
|
|
if (this.chance(0.2)) {
|
|||
|
|
await page.mouse.wheel(0, this.rand(2, 10));
|
|||
|
|
}
|
|||
|
|
} else if (mode === 'fast') {
|
|||
|
|
await page.mouse.move(point.x, point.y);
|
|||
|
|
await this.sleep(this.rand(50, 200));
|
|||
|
|
await page.mouse.down();
|
|||
|
|
await this.sleep(this.rand(20, 50));
|
|||
|
|
await page.mouse.up();
|
|||
|
|
} else {
|
|||
|
|
await page.mouse.move(point.x, point.y);
|
|||
|
|
await page.mouse.down();
|
|||
|
|
await page.mouse.up();
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async type(page: any, text: string, mode: HumanizeMode): Promise<void> {
|
|||
|
|
if (mode === 'off') {
|
|||
|
|
await page.keyboard.type(text);
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
const minDelay = mode === 'full' ? 80 : 30;
|
|||
|
|
const maxDelay = mode === 'full' ? 250 : 80;
|
|||
|
|
for (let i = 0; i < text.length; i++) {
|
|||
|
|
const ch = text[i];
|
|||
|
|
// 5% 错字(仅 full 模式)
|
|||
|
|
if (mode === 'full' && this.chance(0.05)) {
|
|||
|
|
const wrong = String.fromCharCode(ch.charCodeAt(0) + 1);
|
|||
|
|
await page.keyboard.type(wrong);
|
|||
|
|
await this.sleep(this.rand(minDelay, maxDelay));
|
|||
|
|
await page.keyboard.press('Backspace');
|
|||
|
|
await this.sleep(this.rand(minDelay, maxDelay));
|
|||
|
|
}
|
|||
|
|
await page.keyboard.type(ch);
|
|||
|
|
await this.sleep(this.rand(minDelay, maxDelay));
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async hover(page: any, point: { x: number; y: number }, mode: HumanizeMode): Promise<void> {
|
|||
|
|
if (mode === 'full') {
|
|||
|
|
const cursor = this.createCursor(page as any);
|
|||
|
|
await cursor.moveTo(point);
|
|||
|
|
await this.sleep(this.rand(200, 800));
|
|||
|
|
} else {
|
|||
|
|
await page.mouse.move(point.x, point.y);
|
|||
|
|
if (mode === 'fast') await this.sleep(this.rand(50, 200));
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async scroll(page: any, deltaY: number, mode: HumanizeMode): Promise<void> {
|
|||
|
|
if (mode === 'off') {
|
|||
|
|
await page.mouse.wheel(0, deltaY);
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
const steps = mode === 'full' ? Math.ceil(Math.abs(deltaY) / 50) : Math.ceil(Math.abs(deltaY) / 200);
|
|||
|
|
const stepSize = deltaY / steps;
|
|||
|
|
const minD = mode === 'full' ? 50 : 10;
|
|||
|
|
const maxD = mode === 'full' ? 150 : 30;
|
|||
|
|
for (let i = 0; i < steps; i++) {
|
|||
|
|
await page.mouse.wheel(0, stepSize);
|
|||
|
|
await this.sleep(this.rand(minD, maxD));
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private sleep(ms: number): Promise<void> {
|
|||
|
|
return new Promise(r => setTimeout(r, ms));
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest humanizer.service
|
|||
|
|
```
|
|||
|
|
预期:PASS(5 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 8 完成`:humanizer 三档行为可用且可测。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 9:BrowserDaemonService.click/fill/type/snapshot/cookie/state(业务方法)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写 SnapshotRefService 测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/snapshot-ref.test.ts
|
|||
|
|
import { SnapshotRefService } from '../../../../src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.js';
|
|||
|
|
|
|||
|
|
describe('SnapshotRefService', () => {
|
|||
|
|
let svc: SnapshotRefService;
|
|||
|
|
let mockPage: any;
|
|||
|
|
|
|||
|
|
beforeEach(() => {
|
|||
|
|
svc = new SnapshotRefService();
|
|||
|
|
mockPage = {
|
|||
|
|
evaluate: jest.fn().mockResolvedValue([
|
|||
|
|
{ ref: 'e1', tag: 'button', text: 'Login' },
|
|||
|
|
{ ref: 'e2', tag: 'input', text: '' },
|
|||
|
|
]),
|
|||
|
|
};
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('snapshot 调 page.evaluate 注入 data-ai-ref,返回 refs', async () => {
|
|||
|
|
const refs = await svc.snapshot('s1', mockPage);
|
|||
|
|
expect(refs).toEqual([
|
|||
|
|
{ ref: 'e1', tag: 'button', text: 'Login' },
|
|||
|
|
{ ref: 'e2', tag: 'input', text: '' },
|
|||
|
|
]);
|
|||
|
|
expect(mockPage.evaluate).toHaveBeenCalledTimes(1);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('refToSelector 返回 [data-ai-ref="..."]', () => {
|
|||
|
|
expect(svc.refToSelector('e15')).toBe('[data-ai-ref="e15"]');
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 实现 SnapshotRefService**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/service/snapshot-ref.service.ts
|
|||
|
|
import { Provide, Scope, ScopeEnum } from '@midwayjs/core';
|
|||
|
|
|
|||
|
|
export interface RefEntry {
|
|||
|
|
ref: string;
|
|||
|
|
tag: string;
|
|||
|
|
text: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Scope(ScopeEnum.Singleton)
|
|||
|
|
export class SnapshotRefService {
|
|||
|
|
/** 给页面所有可交互元素注入 data-ai-ref,返回 ref 列表 */
|
|||
|
|
async snapshot(_sessionName: string, page: any): Promise<RefEntry[]> {
|
|||
|
|
return page.evaluate(() => {
|
|||
|
|
const interactive = Array.from(document.querySelectorAll(
|
|||
|
|
'a, button, input, select, textarea, [role="button"], [role="link"], [contenteditable="true"]'
|
|||
|
|
));
|
|||
|
|
const out: { ref: string; tag: string; text: string }[] = [];
|
|||
|
|
interactive.forEach((el, i) => {
|
|||
|
|
const ref = `e${i + 1}`;
|
|||
|
|
el.setAttribute('data-ai-ref', ref);
|
|||
|
|
const tag = el.tagName.toLowerCase();
|
|||
|
|
let text = (el as HTMLElement).innerText || (el as HTMLInputElement).value || '';
|
|||
|
|
text = text.trim().slice(0, 80);
|
|||
|
|
out.push({ ref, tag, text });
|
|||
|
|
});
|
|||
|
|
return out;
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
refToSelector(ref: string): string {
|
|||
|
|
return `[data-ai-ref="${ref}"]`;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 跑 snapshot-ref 测试**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest snapshot-ref
|
|||
|
|
```
|
|||
|
|
预期:PASS
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 在 BrowserDaemonService 中加业务方法**
|
|||
|
|
|
|||
|
|
在 `daemon.service.ts` 末尾加入(在 `setHumanizeMode` 之后):
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// ===== 业务方法(依赖 humanizer + snapshot-ref)=====
|
|||
|
|
|
|||
|
|
@Inject() humanizer: HumanizerService;
|
|||
|
|
@Inject() snapshotRef: SnapshotRefService;
|
|||
|
|
|
|||
|
|
async click(sessionName: string, ref: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { page } = this.getPageOrFail(sessionName);
|
|||
|
|
const locator = page.locator(this.snapshotRef.refToSelector(ref));
|
|||
|
|
const box = await locator.boundingBox();
|
|||
|
|
if (!box) throw new Error(`Ref '${ref}' not visible`);
|
|||
|
|
const m = mode ?? this.getHumanizeMode(sessionName);
|
|||
|
|
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async fill(sessionName: string, ref: string, value: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { page } = this.getPageOrFail(sessionName);
|
|||
|
|
const locator = page.locator(this.snapshotRef.refToSelector(ref));
|
|||
|
|
const box = await locator.boundingBox();
|
|||
|
|
if (!box) throw new Error(`Ref '${ref}' not visible`);
|
|||
|
|
const m = mode ?? this.getHumanizeMode(sessionName);
|
|||
|
|
await this.humanizer.click(page, { x: box.x + box.width / 2, y: box.y + box.height / 2 }, m);
|
|||
|
|
await locator.fill('');
|
|||
|
|
await this.humanizer.type(page, value, m);
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async type(sessionName: string, text: string, mode?: 'full' | 'fast' | 'off'): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { page } = this.getPageOrFail(sessionName);
|
|||
|
|
await this.humanizer.type(page, text, mode ?? this.getHumanizeMode(sessionName));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async snapshot(sessionName: string) {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { page } = this.getPageOrFail(sessionName);
|
|||
|
|
const refs = await this.snapshotRef.snapshot(sessionName, page);
|
|||
|
|
return { url: page.url(), refs };
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async getCookies(sessionName: string, domain?: string): Promise<any[]> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { ctx } = this.getPageOrFail(sessionName);
|
|||
|
|
const cookies = await ctx.cookies();
|
|||
|
|
return domain ? cookies.filter((c: any) => c.domain.includes(domain)) : cookies;
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async saveState(sessionName: string, filePath: string): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { ctx } = this.getPageOrFail(sessionName);
|
|||
|
|
await ctx.storageState({ path: filePath });
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async loadState(sessionName: string, filePath: string): Promise<void> {
|
|||
|
|
// loadState 需在 launchPersistentContext 时通过 storageState 选项注入
|
|||
|
|
// 但 patchright launchPersistentContext 不直接接受 storageState
|
|||
|
|
// → 实际方案:addCookies + 手动 localStorage(基于 file 内容)
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { ctx, page } = this.getPageOrFail(sessionName);
|
|||
|
|
const fs = await import('node:fs/promises');
|
|||
|
|
const data = JSON.parse(await fs.readFile(filePath, 'utf8'));
|
|||
|
|
if (data.cookies) await ctx.addCookies(data.cookies);
|
|||
|
|
if (data.origins) {
|
|||
|
|
for (const o of data.origins) {
|
|||
|
|
await page.goto(o.origin);
|
|||
|
|
for (const item of o.localStorage ?? []) {
|
|||
|
|
await page.evaluate(([k, v]: [string, string]) => localStorage.setItem(k, v), [item.name, item.value]);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
async goto(sessionName: string, url: string): Promise<void> {
|
|||
|
|
return this.registry.withLock(sessionName, async () => {
|
|||
|
|
const { page } = this.getPageOrFail(sessionName);
|
|||
|
|
await page.goto(url);
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
也要在文件顶部加 import:
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
import { HumanizerService } from './humanizer.service.js';
|
|||
|
|
import { SnapshotRefService } from './snapshot-ref.service.js';
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 写新方法的测试(追加到 daemon.service.test.ts)**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// 追加到 daemon.service.test.ts
|
|||
|
|
describe('BrowserDaemonService 业务方法', () => {
|
|||
|
|
let svc: BrowserDaemonService;
|
|||
|
|
let mockCtx: any;
|
|||
|
|
let mockPage: any;
|
|||
|
|
let mockHumanizer: any;
|
|||
|
|
let mockSnapshotRef: any;
|
|||
|
|
|
|||
|
|
beforeEach(() => {
|
|||
|
|
mockPage = {
|
|||
|
|
url: () => 'https://example.com',
|
|||
|
|
mouse: { down: jest.fn(), up: jest.fn(), move: jest.fn() },
|
|||
|
|
keyboard: { type: jest.fn() },
|
|||
|
|
locator: jest.fn().mockReturnValue({
|
|||
|
|
boundingBox: jest.fn().mockResolvedValue({ x: 10, y: 20, width: 100, height: 40 }),
|
|||
|
|
fill: jest.fn(),
|
|||
|
|
}),
|
|||
|
|
goto: jest.fn(),
|
|||
|
|
};
|
|||
|
|
mockCtx = {
|
|||
|
|
pages: () => [mockPage],
|
|||
|
|
cookies: jest.fn().mockResolvedValue([{ name: 'a', value: '1', domain: '.x.com' }]),
|
|||
|
|
storageState: jest.fn(),
|
|||
|
|
addCookies: jest.fn(),
|
|||
|
|
};
|
|||
|
|
mockHumanizer = { click: jest.fn(), type: jest.fn() };
|
|||
|
|
mockSnapshotRef = {
|
|||
|
|
refToSelector: (r: string) => `[data-ai-ref="${r}"]`,
|
|||
|
|
snapshot: jest.fn().mockResolvedValue([{ ref: 'e1', tag: 'button', text: 'OK' }]),
|
|||
|
|
};
|
|||
|
|
svc = new BrowserDaemonService();
|
|||
|
|
const { SessionRegistry } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-registry.js');
|
|||
|
|
const { SessionScheduler } = require('../../../../src/modules/netaclaw/browser-daemon/runtime/session-scheduler.js');
|
|||
|
|
(svc as any).registry = new SessionRegistry();
|
|||
|
|
(svc as any).scheduler = new SessionScheduler({ maxActiveSessions: 5 });
|
|||
|
|
(svc as any).humanizer = mockHumanizer;
|
|||
|
|
(svc as any).snapshotRef = mockSnapshotRef;
|
|||
|
|
(svc as any).logger = { info: jest.fn(), warn: jest.fn(), error: jest.fn() };
|
|||
|
|
(svc as any).registry.register('s1', mockCtx);
|
|||
|
|
(svc as any).scheduler.recordActivate('s1');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('click: 调 humanizer.click(中心点, mode)', async () => {
|
|||
|
|
await svc.click('s1', 'e1');
|
|||
|
|
expect(mockHumanizer.click).toHaveBeenCalledWith(mockPage, { x: 60, y: 40 }, 'full');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('snapshot: 调 snapshotRef + 返回 url + refs', async () => {
|
|||
|
|
const r = await svc.snapshot('s1');
|
|||
|
|
expect(r.url).toBe('https://example.com');
|
|||
|
|
expect(r.refs).toEqual([{ ref: 'e1', tag: 'button', text: 'OK' }]);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('getCookies: domain 过滤', async () => {
|
|||
|
|
const r = await svc.getCookies('s1', 'x.com');
|
|||
|
|
expect(r).toEqual([{ name: 'a', value: '1', domain: '.x.com' }]);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('saveState: 调 storageState({path})', async () => {
|
|||
|
|
await svc.saveState('s1', '/tmp/s.json');
|
|||
|
|
expect(mockCtx.storageState).toHaveBeenCalledWith({ path: '/tmp/s.json' });
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: 跑所有 daemon 测试**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest daemon.service
|
|||
|
|
```
|
|||
|
|
预期:PASS(前 5 tests + 4 新 tests = 9)
|
|||
|
|
|
|||
|
|
- [ ] **Step 7: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 9 完成`:BrowserDaemonService 业务方法齐备。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 9b:Cleanup + Lifecycle + Idle-Timeout(**新增 P1 修复**)
|
|||
|
|
|
|||
|
|
**问题**:spec §5.4 要求 @Init 扫 known sessions、@Destroy 优雅关闭 + 5s 超时强制 kill;§6.6 要求 60min idle 自动回收。原 plan 未实现。
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts`
|
|||
|
|
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/service/daemon.service.ts`
|
|||
|
|
- Modify: `packages/backend/src/modules/netaclaw/browser-daemon/runtime/session-scheduler.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 实现 cleanup.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/runtime/cleanup.ts
|
|||
|
|
/**
|
|||
|
|
* 优雅关闭工具:等待 ctx.close() 最多 5s,超时强制 kill
|
|||
|
|
*/
|
|||
|
|
export async function gracefullyClose(ctx: any, timeoutMs = 5000): Promise<void> {
|
|||
|
|
await Promise.race([
|
|||
|
|
ctx.close().catch(() => {}),
|
|||
|
|
new Promise(resolve => setTimeout(resolve, timeoutMs)),
|
|||
|
|
]);
|
|||
|
|
// 强制 kill 残留 chrome 进程
|
|||
|
|
try {
|
|||
|
|
const browser = ctx.browser?.();
|
|||
|
|
if (browser?.process) {
|
|||
|
|
const proc = browser.process();
|
|||
|
|
if (proc && !proc.killed) proc.kill('SIGKILL');
|
|||
|
|
}
|
|||
|
|
} catch {}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: SessionScheduler 增加 idle 计时器**
|
|||
|
|
|
|||
|
|
修改 `session-scheduler.ts`,加入:
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// 在 SessionScheduler 类内
|
|||
|
|
private idleTimers = new Map<string, NodeJS.Timeout>();
|
|||
|
|
private idleTimeoutMs: number;
|
|||
|
|
private onIdleTimeout?: (sessionName: string) => Promise<void>;
|
|||
|
|
|
|||
|
|
constructor(opts: SchedulerOptions = {}) {
|
|||
|
|
this.maxActiveSessions = opts.maxActiveSessions ?? 50;
|
|||
|
|
this.queueTimeoutMs = opts.queueTimeoutMs ?? 30_000;
|
|||
|
|
this.idleTimeoutMs = opts.idleTimeoutMs ?? 60 * 60 * 1000; // 60 min
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
setIdleTimeoutHandler(fn: (sessionName: string) => Promise<void>): void {
|
|||
|
|
this.onIdleTimeout = fn;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
touch(sessionName: string): void {
|
|||
|
|
const s = this.sessions.get(sessionName);
|
|||
|
|
if (s) {
|
|||
|
|
s.status = 'active';
|
|||
|
|
s.lastUsedAt = new Date();
|
|||
|
|
}
|
|||
|
|
this.resetIdleTimer(sessionName);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private resetIdleTimer(sessionName: string): void {
|
|||
|
|
this.clearIdleTimer(sessionName);
|
|||
|
|
if (this.idleTimeoutMs <= 0 || !this.onIdleTimeout) return;
|
|||
|
|
const timer = setTimeout(() => {
|
|||
|
|
this.idleTimers.delete(sessionName);
|
|||
|
|
this.onIdleTimeout?.(sessionName).catch(() => {});
|
|||
|
|
}, this.idleTimeoutMs);
|
|||
|
|
this.idleTimers.set(sessionName, timer);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
private clearIdleTimer(sessionName: string): void {
|
|||
|
|
const t = this.idleTimers.get(sessionName);
|
|||
|
|
if (t) {
|
|||
|
|
clearTimeout(t);
|
|||
|
|
this.idleTimers.delete(sessionName);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
// 在 recordActivate 末尾调 resetIdleTimer
|
|||
|
|
// 在 recordDeactivate 调 clearIdleTimer
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: daemon.service @Init / @Destroy 钩子**
|
|||
|
|
|
|||
|
|
在 `daemon.service.ts` 加:
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
import { Init, Destroy } from '@midwayjs/core';
|
|||
|
|
import { gracefullyClose } from '../runtime/cleanup.js';
|
|||
|
|
import { getStateDir } from '../runtime/browser-data-dir.js';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import * as fs from 'node:fs';
|
|||
|
|
|
|||
|
|
// 在 BrowserDaemonService 类内:
|
|||
|
|
|
|||
|
|
@Init()
|
|||
|
|
async init(): Promise<void> {
|
|||
|
|
// 注册 idle 回收回调
|
|||
|
|
this.scheduler.setIdleTimeoutHandler(async (sessionName) => {
|
|||
|
|
this.logger.info(`[browser-daemon] idle timeout: ${sessionName} → save+close`);
|
|||
|
|
await this.registry.withLock(sessionName, async () => {
|
|||
|
|
const ctx = this.registry.get(sessionName);
|
|||
|
|
if (!ctx) return;
|
|||
|
|
const stateFile = getStateDir(sessionName);
|
|||
|
|
try {
|
|||
|
|
await ctx.storageState({ path: stateFile });
|
|||
|
|
} catch (e: any) {
|
|||
|
|
this.logger.warn(`[browser-daemon] saveState on idle failed: ${e.message}`);
|
|||
|
|
}
|
|||
|
|
await this.closeInternal(sessionName);
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
// 扫描 known sessions(不主动 launch)
|
|||
|
|
const stateRoot = path.dirname(getStateDir('placeholder'));
|
|||
|
|
if (fs.existsSync(stateRoot)) {
|
|||
|
|
const files = fs.readdirSync(stateRoot).filter(f => f.endsWith('.json'));
|
|||
|
|
this.logger.info(`[browser-daemon] init: ${files.length} known sessions on disk`);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Destroy()
|
|||
|
|
async destroy(): Promise<void> {
|
|||
|
|
this.logger.info('[browser-daemon] destroying, closing all sessions...');
|
|||
|
|
const all = this.registry.list();
|
|||
|
|
await Promise.all(all.map(async (e) => {
|
|||
|
|
try {
|
|||
|
|
const stateFile = getStateDir(e.sessionName);
|
|||
|
|
await e.context.storageState({ path: stateFile });
|
|||
|
|
} catch (err: any) {
|
|||
|
|
this.logger.warn(`[browser-daemon] saveState ${e.sessionName} failed: ${err.message}`);
|
|||
|
|
}
|
|||
|
|
await gracefullyClose(e.context, 5000);
|
|||
|
|
}));
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 写 cleanup 测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/cleanup.test.ts
|
|||
|
|
import { gracefullyClose } from '../../../../src/modules/netaclaw/browser-daemon/runtime/cleanup.js';
|
|||
|
|
|
|||
|
|
describe('cleanup.gracefullyClose', () => {
|
|||
|
|
it('正常关闭:close 完成', async () => {
|
|||
|
|
const close = jest.fn().mockResolvedValue(undefined);
|
|||
|
|
await gracefullyClose({ close }, 1000);
|
|||
|
|
expect(close).toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('close 超时:5s 后强制 kill', async () => {
|
|||
|
|
const kill = jest.fn();
|
|||
|
|
const ctx = {
|
|||
|
|
close: () => new Promise(() => {}), // 永远不 resolve
|
|||
|
|
browser: () => ({ process: () => ({ kill, killed: false }) }),
|
|||
|
|
};
|
|||
|
|
const start = Date.now();
|
|||
|
|
await gracefullyClose(ctx, 100);
|
|||
|
|
expect(Date.now() - start).toBeGreaterThanOrEqual(100);
|
|||
|
|
expect(kill).toHaveBeenCalledWith('SIGKILL');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('close 抛错不阻断', async () => {
|
|||
|
|
await expect(
|
|||
|
|
gracefullyClose({ close: () => Promise.reject(new Error('boom')) }, 100)
|
|||
|
|
).resolves.toBeUndefined();
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 跑测试**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest cleanup.test
|
|||
|
|
```
|
|||
|
|
预期:PASS(3 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: SessionScheduler idle timer 测试追加**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// 追加到 session-scheduler.test.ts
|
|||
|
|
describe('SessionScheduler idle timer', () => {
|
|||
|
|
it('idle 超时后调 onIdleTimeout', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 50 });
|
|||
|
|
const handler = jest.fn().mockResolvedValue(undefined);
|
|||
|
|
sch.setIdleTimeoutHandler(handler);
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
await new Promise(r => setTimeout(r, 100));
|
|||
|
|
expect(handler).toHaveBeenCalledWith('s1');
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('touch 重置计时器', async () => {
|
|||
|
|
const sch = new SessionScheduler({ maxActiveSessions: 5, idleTimeoutMs: 80 });
|
|||
|
|
const handler = jest.fn().mockResolvedValue(undefined);
|
|||
|
|
sch.setIdleTimeoutHandler(handler);
|
|||
|
|
sch.recordActivate('s1');
|
|||
|
|
await new Promise(r => setTimeout(r, 50));
|
|||
|
|
sch.touch('s1');
|
|||
|
|
await new Promise(r => setTimeout(r, 50));
|
|||
|
|
// 累积 100ms 但 touch 重置后只过了 50ms,未触发
|
|||
|
|
expect(handler).not.toHaveBeenCalled();
|
|||
|
|
await new Promise(r => setTimeout(r, 50));
|
|||
|
|
expect(handler).toHaveBeenCalledWith('s1');
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 7: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 9b 完成`:lifecycle 钩子 + idle 自动回收 + 优雅关闭。验收 9 + 5b 可通过。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 10:control-auth.middleware(loopback + secret)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts`
|
|||
|
|
- Test: `packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写测试**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/control-auth.middleware.test.ts
|
|||
|
|
import { BrowserControlAuthMiddleware } from '../../../../src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.js';
|
|||
|
|
|
|||
|
|
describe('BrowserControlAuthMiddleware', () => {
|
|||
|
|
const mw = new BrowserControlAuthMiddleware();
|
|||
|
|
const fn = mw.resolve();
|
|||
|
|
|
|||
|
|
beforeEach(() => {
|
|||
|
|
process.env.NETA_TRAY_SECRET = 'test-secret';
|
|||
|
|
process.env.NODE_ENV = 'test';
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('loopback + 正确 secret → 通过', async () => {
|
|||
|
|
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(next).toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('非 loopback → 403', async () => {
|
|||
|
|
const ctx: any = { ip: '8.8.8.8', headers: { 'x-neta-control-secret': 'test-secret' }, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(ctx.status).toBe(403);
|
|||
|
|
expect(next).not.toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('loopback + 错误 secret → 401', async () => {
|
|||
|
|
const ctx: any = { ip: '127.0.0.1', headers: { 'x-neta-control-secret': 'wrong' }, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(ctx.status).toBe(401);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('loopback 无 secret → 401', async () => {
|
|||
|
|
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(ctx.status).toBe(401);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('dev 模式 + secret 未配置 → 旁路通过(loopback 仍校验)', async () => {
|
|||
|
|
process.env.NETA_TRAY_SECRET = '';
|
|||
|
|
process.env.NODE_ENV = 'development';
|
|||
|
|
const ctx: any = { ip: '127.0.0.1', headers: {}, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(next).toHaveBeenCalled();
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
it('dev 模式 + secret 未配置 + 非 loopback → 仍 403', async () => {
|
|||
|
|
process.env.NETA_TRAY_SECRET = '';
|
|||
|
|
process.env.NODE_ENV = 'development';
|
|||
|
|
const ctx: any = { ip: '8.8.8.8', headers: {}, status: 200 };
|
|||
|
|
const next = jest.fn();
|
|||
|
|
await fn(ctx, next);
|
|||
|
|
expect(ctx.status).toBe(403);
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑测试看失败**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest control-auth.middleware
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/middleware/control-auth.middleware.ts
|
|||
|
|
import { IMiddleware, Middleware } from '@midwayjs/core';
|
|||
|
|
import { NextFunction, Context } from '@midwayjs/koa';
|
|||
|
|
import { isLoopbackAddress, validateRuntimeSecret } from '../../../../comm/runtime-secret.js';
|
|||
|
|
|
|||
|
|
@Middleware()
|
|||
|
|
export class BrowserControlAuthMiddleware implements IMiddleware<Context, NextFunction> {
|
|||
|
|
resolve() {
|
|||
|
|
return async (ctx: Context, next: NextFunction) => {
|
|||
|
|
const ip = ctx.ip || (ctx as any).request?.ip || (ctx as any).req?.socket?.remoteAddress;
|
|||
|
|
if (!isLoopbackAddress(ip)) {
|
|||
|
|
ctx.status = 403;
|
|||
|
|
ctx.body = { code: 1003, error: 'Forbidden: only loopback access allowed' };
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
const expected = process.env.NETA_TRAY_SECRET ?? '';
|
|||
|
|
const actual = ctx.headers['x-neta-control-secret'] as string | undefined;
|
|||
|
|
const isDev = process.env.NODE_ENV !== 'production';
|
|||
|
|
// dev 旁路:secret 未配置时仅信任 loopback
|
|||
|
|
if (isDev && !expected) {
|
|||
|
|
await next();
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
if (!validateRuntimeSecret(expected, actual)) {
|
|||
|
|
ctx.status = 401;
|
|||
|
|
ctx.body = { code: 1001, error: 'Unauthorized: invalid x-neta-control-secret' };
|
|||
|
|
return;
|
|||
|
|
}
|
|||
|
|
await next();
|
|||
|
|
};
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
match(ctx: Context): boolean {
|
|||
|
|
return ctx.path?.startsWith('/admin/browser-daemon/');
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
static getName(): string {
|
|||
|
|
return 'browserControlAuth';
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 跑测试看通过**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest control-auth.middleware
|
|||
|
|
```
|
|||
|
|
预期:PASS(6 tests)
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 在 netaclaw/config.ts 注册中间件(重要:模块挂载明确)**
|
|||
|
|
|
|||
|
|
修改 `packages/backend/src/modules/netaclaw/config.ts`,把空的 `middlewares: []` 改为:
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
import { ModuleConfig } from '@cool-midway/core';
|
|||
|
|
import { BrowserControlAuthMiddleware } from './browser-daemon/middleware/control-auth.middleware.js';
|
|||
|
|
|
|||
|
|
export default () => {
|
|||
|
|
return {
|
|||
|
|
name: 'NetaClaw',
|
|||
|
|
description: 'NetaClaw 电商浏览器自动化 Agent 引擎',
|
|||
|
|
middlewares: [BrowserControlAuthMiddleware],
|
|||
|
|
globalMiddlewares: [],
|
|||
|
|
order: 0,
|
|||
|
|
} as ModuleConfig;
|
|||
|
|
};
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> 关键:browser-daemon **不是独立模块**,是 netaclaw 的子目录。中间件挂载到 netaclaw 模块,靠 `match()` 限制只在 `/admin/browser-daemon/*` 路径生效。其他 netaclaw 路由(`/admin/netaclaw/*`)不受影响。
|
|||
|
|
|
|||
|
|
注意保留 NetaClawConfig / defaultNetaClawConfig 等导出(在 `// ---` 分隔符之后的部分)。
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 10 完成`:auth middleware 校验 loopback + secret,含 dev 旁路;middleware 已挂载到 netaclaw 模块。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 11:HTTP Controllers(5 个 controller)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts`
|
|||
|
|
- Create: `packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts`
|
|||
|
|
|
|||
|
|
> 这 5 个 controller 都是 thin wrapper:解析 body → 调 service → 返回结果。**严格按 Neta 现有 controller 风格**:
|
|||
|
|
> - `@Provide()` + `@Controller('/admin/browser-daemon')` 类装饰器声明路径前缀
|
|||
|
|
> - 方法用相对路径 `@Post('/open')` `@Get('/list')`
|
|||
|
|
> - 直接返回 `{ code: 1000, data: ... }`,不继承 `BaseController`,不用 `this.ok()`,不用 `@CoolController` 不用 `@CoolTag`
|
|||
|
|
> - 参考 `packages/backend/src/modules/netaclaw/controller/admin/agent_channel.ts` 实际风格
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 实现 session.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/session.ts
|
|||
|
|
import { Provide, Inject, Controller, Post, Get, Body } from '@midwayjs/core';
|
|||
|
|
import { Context } from '@midwayjs/koa';
|
|||
|
|
import { BrowserDaemonService } from '../../service/daemon.service.js';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Controller('/admin/browser-daemon')
|
|||
|
|
export class AdminBrowserDaemonSessionController {
|
|||
|
|
@Inject() daemonService: BrowserDaemonService;
|
|||
|
|
@Inject() ctx: Context;
|
|||
|
|
|
|||
|
|
@Post('/open')
|
|||
|
|
async open(@Body() dto: any) {
|
|||
|
|
try {
|
|||
|
|
const r = await this.daemonService.open(dto);
|
|||
|
|
return { code: 1000, data: r };
|
|||
|
|
} catch (e: any) {
|
|||
|
|
this.ctx.status = e.statusCode ?? 400;
|
|||
|
|
return { code: 1002, error: e.message, retryAfter: e.retryAfter };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Post('/close')
|
|||
|
|
async close(@Body('sessionName') sessionName: string) {
|
|||
|
|
await this.daemonService.close(sessionName);
|
|||
|
|
return { code: 1000, data: { ok: true } };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Get('/list')
|
|||
|
|
async list() {
|
|||
|
|
return { code: 1000, data: this.daemonService.list() };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Get('/stats')
|
|||
|
|
async stats() {
|
|||
|
|
return { code: 1000, data: this.daemonService.getStats() };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 实现 interaction.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/interaction.ts
|
|||
|
|
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
|
|||
|
|
import { BrowserDaemonService } from '../../service/daemon.service.js';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Controller('/admin/browser-daemon')
|
|||
|
|
export class AdminBrowserDaemonInteractionController {
|
|||
|
|
@Inject() daemonService: BrowserDaemonService;
|
|||
|
|
|
|||
|
|
@Post('/click')
|
|||
|
|
async click(@Body() dto: { sessionName: string; ref: string; mode?: 'full' | 'fast' | 'off' }) {
|
|||
|
|
await this.daemonService.click(dto.sessionName, dto.ref, dto.mode ?? 'full');
|
|||
|
|
return { code: 1000, data: { clicked: dto.ref } };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Post('/fill')
|
|||
|
|
async fill(@Body() dto: { sessionName: string; ref: string; value: string; mode?: 'full' | 'fast' | 'off' }) {
|
|||
|
|
await this.daemonService.fill(dto.sessionName, dto.ref, dto.value, dto.mode ?? 'full');
|
|||
|
|
return { code: 1000, data: { filled: dto.ref } };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Post('/type')
|
|||
|
|
async type(@Body() dto: { sessionName: string; text: string; mode?: 'full' | 'fast' | 'off' }) {
|
|||
|
|
await this.daemonService.type(dto.sessionName, dto.text, dto.mode ?? 'full');
|
|||
|
|
return { code: 1000, data: { typed: dto.text.length } };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 实现 navigation.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/navigation.ts
|
|||
|
|
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
|
|||
|
|
import { BrowserDaemonService } from '../../service/daemon.service.js';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Controller('/admin/browser-daemon')
|
|||
|
|
export class AdminBrowserDaemonNavigationController {
|
|||
|
|
@Inject() daemonService: BrowserDaemonService;
|
|||
|
|
|
|||
|
|
@Post('/goto')
|
|||
|
|
async goto(@Body() dto: { sessionName: string; url: string }) {
|
|||
|
|
await this.daemonService.goto(dto.sessionName, dto.url);
|
|||
|
|
return { code: 1000, data: { navigated: dto.url } };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 实现 state.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/state.ts
|
|||
|
|
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
|
|||
|
|
import { BrowserDaemonService } from '../../service/daemon.service.js';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Controller('/admin/browser-daemon')
|
|||
|
|
export class AdminBrowserDaemonStateController {
|
|||
|
|
@Inject() daemonService: BrowserDaemonService;
|
|||
|
|
|
|||
|
|
@Post('/cookie-list')
|
|||
|
|
async cookieList(@Body() dto: { sessionName: string; domain?: string }) {
|
|||
|
|
return { code: 1000, data: await this.daemonService.getCookies(dto.sessionName, dto.domain) };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Post('/state-save')
|
|||
|
|
async stateSave(@Body() dto: { sessionName: string; filePath: string }) {
|
|||
|
|
await this.daemonService.saveState(dto.sessionName, dto.filePath);
|
|||
|
|
return { code: 1000, data: { saved: dto.filePath } };
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
@Post('/state-load')
|
|||
|
|
async stateLoad(@Body() dto: { sessionName: string; filePath: string }) {
|
|||
|
|
await this.daemonService.loadState(dto.sessionName, dto.filePath);
|
|||
|
|
return { code: 1000, data: { loaded: dto.filePath } };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: 实现 inspect.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/src/modules/netaclaw/browser-daemon/controller/admin/inspect.ts
|
|||
|
|
import { Provide, Inject, Controller, Post, Body } from '@midwayjs/core';
|
|||
|
|
import { BrowserDaemonService } from '../../service/daemon.service.js';
|
|||
|
|
|
|||
|
|
@Provide()
|
|||
|
|
@Controller('/admin/browser-daemon')
|
|||
|
|
export class AdminBrowserDaemonInspectController {
|
|||
|
|
@Inject() daemonService: BrowserDaemonService;
|
|||
|
|
|
|||
|
|
@Post('/snapshot')
|
|||
|
|
async snapshot(@Body() dto: { sessionName: string }) {
|
|||
|
|
return { code: 1000, data: await this.daemonService.snapshot(dto.sessionName) };
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: 启动 backend 验证路由**
|
|||
|
|
|
|||
|
|
dev 模式启动(dev 旁路时无需 secret):
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
NODE_ENV=development pnpm dev
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
另一终端:
|
|||
|
|
```bash
|
|||
|
|
# dev 旁路(middleware 已实现):仅 loopback 即可,secret 未设也能调
|
|||
|
|
curl -X GET http://localhost:8003/admin/browser-daemon/list
|
|||
|
|
```
|
|||
|
|
预期:200 + `{"code":1000,"data":[]}`
|
|||
|
|
|
|||
|
|
- [ ] **Step 7: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 11 完成`:5 个 controller 上线,HTTP API 可达。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 12:Contract Test(HTTP ↔ service 等价性)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 写 contract test**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/backend/test/modules/netaclaw/browser-daemon/contract.test.ts
|
|||
|
|
import { createApp, close } from '@midwayjs/mock';
|
|||
|
|
import { Framework } from '@midwayjs/koa';
|
|||
|
|
|
|||
|
|
describe('BrowserDaemon HTTP ↔ Service 契约', () => {
|
|||
|
|
let app: any;
|
|||
|
|
beforeAll(async () => {
|
|||
|
|
process.env.NETA_TRAY_SECRET = 'ct-secret';
|
|||
|
|
app = await createApp(undefined, undefined, Framework);
|
|||
|
|
});
|
|||
|
|
afterAll(async () => { if (app) await close(app); });
|
|||
|
|
|
|||
|
|
/** Mock daemonService,验证 controller 调它的方法名/参数与 HTTP 1:1 */
|
|||
|
|
const cases = [
|
|||
|
|
{ http: { method: 'POST', path: '/admin/browser-daemon/close', body: { sessionName: 's1' } }, service: 'close', args: ['s1'] },
|
|||
|
|
{ http: { method: 'POST', path: '/admin/browser-daemon/click', body: { sessionName: 's1', ref: 'e1', mode: 'full' } }, service: 'click', args: ['s1', 'e1', 'full'] },
|
|||
|
|
{ http: { method: 'POST', path: '/admin/browser-daemon/fill', body: { sessionName: 's1', ref: 'e1', value: 'abc' } }, service: 'fill', args: ['s1', 'e1', 'abc', undefined] },
|
|||
|
|
{ http: { method: 'POST', path: '/admin/browser-daemon/cookie-list', body: { sessionName: 's1', domain: 'x.com' } }, service: 'getCookies', args: ['s1', 'x.com'] },
|
|||
|
|
{ http: { method: 'POST', path: '/admin/browser-daemon/snapshot', body: { sessionName: 's1' } }, service: 'snapshot', args: ['s1'] },
|
|||
|
|
];
|
|||
|
|
|
|||
|
|
for (const c of cases) {
|
|||
|
|
it(`HTTP ${c.http.method} ${c.http.path} → service.${c.service}(${c.args.join(',')})`, async () => {
|
|||
|
|
const fakeService = {
|
|||
|
|
[c.service]: jest.fn().mockResolvedValue(undefined),
|
|||
|
|
};
|
|||
|
|
// 替换 daemonService 实例
|
|||
|
|
const container = (app as any).getApplicationContext();
|
|||
|
|
container.registerObject('browserDaemonService', fakeService);
|
|||
|
|
// 调 HTTP
|
|||
|
|
const supertest = require('supertest');
|
|||
|
|
const res = await supertest(app.getServer())[c.http.method.toLowerCase()](c.http.path)
|
|||
|
|
.set('x-neta-control-secret', 'ct-secret')
|
|||
|
|
.send(c.http.body);
|
|||
|
|
expect(res.status).toBe(200);
|
|||
|
|
expect(fakeService[c.service]).toHaveBeenCalledWith(...c.args);
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
> 注:实施时依据 Midway test 实际 API 调整 `registerObject` / `supertest` 接入;目的是验证 HTTP 与 service 一一对应。
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 跑 contract test**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pnpm jest contract.test
|
|||
|
|
```
|
|||
|
|
预期:PASS(5 cases)
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 12 完成`:HTTP 与 service 等价性有 contract test 兜底。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 13-18:netabrowser-cli 包实现(合并描述)
|
|||
|
|
|
|||
|
|
> 这 6 个 task 是 CLI 包内部各文件,模式相似(接受参数 → 调 HTTP → 输出)。每个文件按下面同一套结构实现,不再展开每 step。
|
|||
|
|
|
|||
|
|
### Task 13:bin/main.ts(commander 解析 + 命令分发)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/netabrowser-cli/src/bin/main.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 实现**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
#!/usr/bin/env node
|
|||
|
|
// packages/netabrowser-cli/src/bin/main.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { registerSessionCommands } from '../commands/session.js';
|
|||
|
|
import { registerInteractionCommands } from '../commands/interaction.js';
|
|||
|
|
import { registerNavigationCommands } from '../commands/navigation.js';
|
|||
|
|
import { registerStateCommands } from '../commands/state.js';
|
|||
|
|
import { registerInspectCommands } from '../commands/inspect.js';
|
|||
|
|
|
|||
|
|
const program = new Command();
|
|||
|
|
program
|
|||
|
|
.name('netabrowser-cli')
|
|||
|
|
.description('Neta 反风控+拟人化浏览器 CLI')
|
|||
|
|
.version('0.1.0');
|
|||
|
|
|
|||
|
|
registerSessionCommands(program);
|
|||
|
|
registerInteractionCommands(program);
|
|||
|
|
registerNavigationCommands(program);
|
|||
|
|
registerStateCommands(program);
|
|||
|
|
registerInspectCommands(program);
|
|||
|
|
|
|||
|
|
program.parseAsync(process.argv).catch(e => {
|
|||
|
|
console.error('Error:', e.message);
|
|||
|
|
process.exit(1);
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: Checkpoint**
|
|||
|
|
|
|||
|
|
### Task 14:client/runtime-info + http-client
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/netabrowser-cli/src/client/runtime-info.ts`
|
|||
|
|
- Create: `packages/netabrowser-cli/src/client/http-client.ts`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: runtime-info.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/client/runtime-info.ts
|
|||
|
|
import * as fs from 'node:fs';
|
|||
|
|
import * as path from 'node:path';
|
|||
|
|
import * as os from 'node:os';
|
|||
|
|
|
|||
|
|
export interface RuntimeInfoLite {
|
|||
|
|
port: number;
|
|||
|
|
controlSecret: string;
|
|||
|
|
controlBaseUrl: string;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
/**
|
|||
|
|
* v2 修复 P1-6:CLI 启动时 cwd 不一定 = backend cwd。
|
|||
|
|
* runtime-info.json 真实写入 packages/backend/dist/runtime-info.json。
|
|||
|
|
* 优先级:环境变量 > pkg 同目录 > monorepo backend dist > user home。
|
|||
|
|
*/
|
|||
|
|
export function readRuntimeInfo(): RuntimeInfoLite {
|
|||
|
|
if (process.env.NETA_RUNTIME_INFO) {
|
|||
|
|
return JSON.parse(fs.readFileSync(process.env.NETA_RUNTIME_INFO, 'utf8'));
|
|||
|
|
}
|
|||
|
|
const candidates = [
|
|||
|
|
// pkg 模式:netabrowser-cli.exe 同目录 data/runtime-info.json
|
|||
|
|
path.join(path.dirname(process.execPath), 'data', 'runtime-info.json'),
|
|||
|
|
// dev 模式:从 cli 编译产物上溯找 monorepo,再进 backend dist
|
|||
|
|
// __dirname = <root>/packages/netabrowser-cli/dist/client
|
|||
|
|
// 上溯 4 层到 monorepo root,再进 packages/backend/dist
|
|||
|
|
path.resolve(__dirname, '../../../../../packages/backend/dist/runtime-info.json'),
|
|||
|
|
// 兜底:用户主目录
|
|||
|
|
path.join(os.homedir(), '.neta', 'runtime-info.json'),
|
|||
|
|
];
|
|||
|
|
for (const p of candidates) {
|
|||
|
|
if (fs.existsSync(p)) return JSON.parse(fs.readFileSync(p, 'utf8'));
|
|||
|
|
}
|
|||
|
|
throw new Error(
|
|||
|
|
`runtime-info.json not found. Set NETA_RUNTIME_INFO env var or run backend first.\nTried:\n${candidates.map(c => ' - ' + c).join('\n')}`
|
|||
|
|
);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: http-client.ts**
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/client/http-client.ts
|
|||
|
|
import axios, { AxiosInstance } from 'axios';
|
|||
|
|
import { readRuntimeInfo } from './runtime-info.js';
|
|||
|
|
|
|||
|
|
let client: AxiosInstance | null = null;
|
|||
|
|
|
|||
|
|
export function getHttpClient(): AxiosInstance {
|
|||
|
|
if (client) return client;
|
|||
|
|
const info = readRuntimeInfo();
|
|||
|
|
client = axios.create({
|
|||
|
|
baseURL: info.controlBaseUrl ?? `http://127.0.0.1:${info.port}`,
|
|||
|
|
headers: { 'x-neta-control-secret': info.controlSecret },
|
|||
|
|
timeout: 60_000,
|
|||
|
|
});
|
|||
|
|
return client;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export async function callDaemon<T = any>(path: string, body?: any): Promise<T> {
|
|||
|
|
const c = getHttpClient();
|
|||
|
|
const res = await c.post(path, body ?? {});
|
|||
|
|
if (res.data?.code !== 1000) {
|
|||
|
|
throw new Error(`Daemon error: ${res.data?.message ?? 'unknown'}`);
|
|||
|
|
}
|
|||
|
|
return res.data.data as T;
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: Checkpoint**
|
|||
|
|
|
|||
|
|
### Task 15:commands/session.ts(open/close/list)
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/commands/session.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { callDaemon } from '../client/http-client.js';
|
|||
|
|
import { formatOutput } from '../output/formatter.js';
|
|||
|
|
|
|||
|
|
export function registerSessionCommands(program: Command) {
|
|||
|
|
program
|
|||
|
|
.command('open <url>')
|
|||
|
|
.description('启动会话并打开 URL')
|
|||
|
|
.requiredOption('--session <name>', 'session 名称')
|
|||
|
|
.option('--proxy <url>', '代理 URL(http://user:pass@host:port)')
|
|||
|
|
.option('--fingerprint-seed <n>', '指纹 seed', parseInt)
|
|||
|
|
.option('--profile-dir <name>', 'profile 目录名')
|
|||
|
|
.option('--headed', '有头模式')
|
|||
|
|
.option('--humanize-mode <mode>', 'full|fast|off', 'full')
|
|||
|
|
.option('--priority <p>', 'low|normal|high', 'normal')
|
|||
|
|
.option('--queue', '排队等位')
|
|||
|
|
.option('--raw', '输出 raw JSON')
|
|||
|
|
.action(async (url, opts) => {
|
|||
|
|
const dto: any = {
|
|||
|
|
sessionName: opts.session,
|
|||
|
|
url,
|
|||
|
|
humanizeMode: opts.humanizeMode,
|
|||
|
|
priority: opts.priority,
|
|||
|
|
queue: !!opts.queue,
|
|||
|
|
headed: !!opts.headed,
|
|||
|
|
};
|
|||
|
|
if (opts.fingerprintSeed != null) dto.fingerprintSeed = opts.fingerprintSeed;
|
|||
|
|
if (opts.profileDir) dto.profileDir = opts.profileDir;
|
|||
|
|
if (opts.proxy) dto.proxy = parseProxyUrl(opts.proxy);
|
|||
|
|
const r = await callDaemon('/admin/browser-daemon/open', dto);
|
|||
|
|
console.log(formatOutput(r, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('close')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
await callDaemon('/admin/browser-daemon/close', { sessionName: opts.session });
|
|||
|
|
console.log(formatOutput({ ok: true }, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('list')
|
|||
|
|
.description('列出所有会话')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
const r = await callDaemon('/admin/browser-daemon/list');
|
|||
|
|
console.log(formatOutput(r, opts.raw));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
function parseProxyUrl(url: string) {
|
|||
|
|
const u = new URL(url);
|
|||
|
|
return {
|
|||
|
|
server: `${u.protocol}//${u.host}`,
|
|||
|
|
username: decodeURIComponent(u.username) || undefined,
|
|||
|
|
password: decodeURIComponent(u.password) || undefined,
|
|||
|
|
};
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Checkpoint**
|
|||
|
|
|
|||
|
|
### Task 16:commands/interaction.ts
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/commands/interaction.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { callDaemon } from '../client/http-client.js';
|
|||
|
|
import { formatOutput } from '../output/formatter.js';
|
|||
|
|
|
|||
|
|
export function registerInteractionCommands(program: Command) {
|
|||
|
|
program
|
|||
|
|
.command('click <ref>')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--mode <mode>', 'full|fast|off')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (ref, opts) => {
|
|||
|
|
// v2 修复 P1-8:CLI 端 mode 优先级 = 命令级 --mode > NETA_BROWSER_HUMANIZE_MODE 环境变量 > 默认 full
|
|||
|
|
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
|
|||
|
|
await callDaemon('/admin/browser-daemon/click', { sessionName: opts.session, ref, mode });
|
|||
|
|
console.log(formatOutput({ clicked: ref }, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('fill <ref> <value>')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--mode <mode>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (ref, value, opts) => {
|
|||
|
|
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
|
|||
|
|
await callDaemon('/admin/browser-daemon/fill', { sessionName: opts.session, ref, value, mode });
|
|||
|
|
console.log(formatOutput({ filled: ref }, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('type <text>')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--mode <mode>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (text, opts) => {
|
|||
|
|
const mode = opts.mode ?? process.env.NETA_BROWSER_HUMANIZE_MODE ?? 'full';
|
|||
|
|
await callDaemon('/admin/browser-daemon/type', { sessionName: opts.session, text, mode });
|
|||
|
|
console.log(formatOutput({ typed: text.length }, opts.raw));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Checkpoint**
|
|||
|
|
|
|||
|
|
### Task 17:commands/navigation + state + inspect
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/commands/navigation.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { callDaemon } from '../client/http-client.js';
|
|||
|
|
import { formatOutput } from '../output/formatter.js';
|
|||
|
|
|
|||
|
|
export function registerNavigationCommands(program: Command) {
|
|||
|
|
program
|
|||
|
|
.command('goto <url>')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (url, opts) => {
|
|||
|
|
await callDaemon('/admin/browser-daemon/goto', { sessionName: opts.session, url });
|
|||
|
|
console.log(formatOutput({ navigated: url }, opts.raw));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/commands/state.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { callDaemon } from '../client/http-client.js';
|
|||
|
|
import { formatOutput } from '../output/formatter.js';
|
|||
|
|
|
|||
|
|
export function registerStateCommands(program: Command) {
|
|||
|
|
program
|
|||
|
|
.command('cookie-list')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--domain <d>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
const r = await callDaemon('/admin/browser-daemon/cookie-list', { sessionName: opts.session, domain: opts.domain });
|
|||
|
|
console.log(formatOutput(r, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('state-save')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.requiredOption('--output <path>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
await callDaemon('/admin/browser-daemon/state-save', { sessionName: opts.session, filePath: opts.output });
|
|||
|
|
console.log(formatOutput({ saved: opts.output }, opts.raw));
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
program
|
|||
|
|
.command('state-load')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.requiredOption('--input <path>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
await callDaemon('/admin/browser-daemon/state-load', { sessionName: opts.session, filePath: opts.input });
|
|||
|
|
console.log(formatOutput({ loaded: opts.input }, opts.raw));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/commands/inspect.ts
|
|||
|
|
import { Command } from 'commander';
|
|||
|
|
import { callDaemon } from '../client/http-client.js';
|
|||
|
|
import { formatOutput } from '../output/formatter.js';
|
|||
|
|
|
|||
|
|
export function registerInspectCommands(program: Command) {
|
|||
|
|
program
|
|||
|
|
.command('snapshot')
|
|||
|
|
.requiredOption('--session <name>')
|
|||
|
|
.option('--raw')
|
|||
|
|
.action(async (opts) => {
|
|||
|
|
const r = await callDaemon('/admin/browser-daemon/snapshot', { sessionName: opts.session });
|
|||
|
|
console.log(formatOutput(r, opts.raw));
|
|||
|
|
});
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Checkpoint**
|
|||
|
|
|
|||
|
|
### Task 18:output/formatter.ts
|
|||
|
|
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/src/output/formatter.ts
|
|||
|
|
export function formatOutput(data: any, raw?: boolean): string {
|
|||
|
|
if (raw) return JSON.stringify(data);
|
|||
|
|
if (Array.isArray(data) && data.length > 0 && data[0]?.ref) {
|
|||
|
|
// snapshot 友好输出
|
|||
|
|
return data.map((r: any) => `[${r.ref}] ${r.tag}: ${r.text}`).join('\n');
|
|||
|
|
}
|
|||
|
|
return JSON.stringify(data, null, 2);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
写最小测试:
|
|||
|
|
```typescript
|
|||
|
|
// packages/netabrowser-cli/tests/output-formatter.test.ts
|
|||
|
|
import { formatOutput } from '../src/output/formatter.js';
|
|||
|
|
|
|||
|
|
describe('formatter', () => {
|
|||
|
|
it('--raw → 单行 JSON', () => {
|
|||
|
|
expect(formatOutput({ a: 1 }, true)).toBe('{"a":1}');
|
|||
|
|
});
|
|||
|
|
it('snapshot refs → 友好输出', () => {
|
|||
|
|
const r = formatOutput([{ ref: 'e1', tag: 'button', text: 'OK' }]);
|
|||
|
|
expect(r).toContain('[e1] button: OK');
|
|||
|
|
});
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step:Checkpoint**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 19:SKILL.md(NetaClaw Agent skill 元数据)
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/skills/netabrowser-cli/SKILL.md`
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
---
|
|||
|
|
name: netabrowser-cli
|
|||
|
|
description: Neta 反风控+拟人化浏览器自动化 CLI。养号、电商自动化、AI 探索复杂网页时使用。比 playwright-cli 更难被反风控识别(fingerprint-chromium 内核 + patchright 反自动化 + ghost-cursor 拟人化轨迹)。100 账号矩阵养号场景首选。
|
|||
|
|
allowed-tools: Bash(netabrowser-cli:*) Bash(npx:*)
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Browser Automation with netabrowser-cli
|
|||
|
|
|
|||
|
|
## When to use vs playwright-cli vs patchwright-cli
|
|||
|
|
|
|||
|
|
- **netabrowser-cli**:国内反风控严的站(小红书/抖音/淘宝/拼多多/微博)+ 多账号矩阵 + 养号场景
|
|||
|
|
- **patchwright-cli**:国外 Cloudflare/DataDome 等反爬保护的站
|
|||
|
|
- **playwright-cli**:测试、无反风控的站
|
|||
|
|
|
|||
|
|
If targeting Chinese social/e-commerce sites or running multi-account automation, **prefer netabrowser-cli**.
|
|||
|
|
|
|||
|
|
## Quick start
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 启动会话(每会话独立指纹+独立 IP)
|
|||
|
|
netabrowser-cli open https://www.xiaohongshu.com \
|
|||
|
|
--session=acc1 \
|
|||
|
|
--proxy=http://user:pass@host:port \
|
|||
|
|
--fingerprint-seed=12345 \
|
|||
|
|
--headed
|
|||
|
|
|
|||
|
|
# 拿当前页面可交互元素
|
|||
|
|
netabrowser-cli snapshot --session=acc1
|
|||
|
|
# 输出:[e1] button: 登录 / [e2] input: 手机号 / ...
|
|||
|
|
|
|||
|
|
# 拟人化点击 + 输入(默认 humanize-mode=full)
|
|||
|
|
netabrowser-cli click e1 --session=acc1
|
|||
|
|
netabrowser-cli fill e2 13800001234 --session=acc1
|
|||
|
|
|
|||
|
|
# 抓 cookie
|
|||
|
|
netabrowser-cli cookie-list --session=acc1 --domain=xiaohongshu.com
|
|||
|
|
|
|||
|
|
# 保存登录态
|
|||
|
|
netabrowser-cli state-save --session=acc1 --output=/path/to/state.json
|
|||
|
|
|
|||
|
|
# 关闭
|
|||
|
|
netabrowser-cli close --session=acc1
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Commands
|
|||
|
|
|
|||
|
|
### Session 管理
|
|||
|
|
- `open <url>` 启动会话
|
|||
|
|
- `close` 关闭
|
|||
|
|
- `list` 列出所有 session
|
|||
|
|
|
|||
|
|
### 交互(默认拟人化 full)
|
|||
|
|
- `click <ref>` 点击 ref 元素(贝塞尔轨迹+视觉停顿+mousedown/up)
|
|||
|
|
- `fill <ref> <value>` focus + 拟人化输入
|
|||
|
|
- `type <text>` 在已 focused 输入框打字
|
|||
|
|
- `--mode=fast` 单命令切批量模式(200-500ms 延迟,无轨迹)
|
|||
|
|
- `--mode=off` 测试用,立即执行
|
|||
|
|
|
|||
|
|
### 导航
|
|||
|
|
- `goto <url>` 跳转
|
|||
|
|
|
|||
|
|
### 状态
|
|||
|
|
- `cookie-list [--domain=x]`
|
|||
|
|
- `state-save --output=path` 保存完整登录态(cookie+localStorage)
|
|||
|
|
- `state-load --input=path` 恢复
|
|||
|
|
|
|||
|
|
### 检查
|
|||
|
|
- `snapshot` 拿可交互元素 ref 列表
|
|||
|
|
|
|||
|
|
## 参数规则
|
|||
|
|
|
|||
|
|
### 指纹
|
|||
|
|
- `--fingerprint-seed=N` 单一 seed 派生所有指纹维度(推荐)
|
|||
|
|
- 不同 seed → 不同 canvas/webgl/UA fingerprint,同 seed → 完全可复现
|
|||
|
|
|
|||
|
|
### 代理
|
|||
|
|
- `--proxy=http://user:pass@host:port` 单参数最方便
|
|||
|
|
- 出口 IP 实测 = 代理 IP(patchright `launchPersistentContext` 验证通过)
|
|||
|
|
|
|||
|
|
### 拟人化档位
|
|||
|
|
| `--humanize-mode` | 单命令开销 | 适用 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| full(默认) | 2-5s | 养号、敏感操作、AI 探索 |
|
|||
|
|
| fast | 0.3-0.7s | 批量发布、批量评论 |
|
|||
|
|
| off | <100ms | 测试、CI |
|
|||
|
|
|
|||
|
|
### 调度
|
|||
|
|
- `--priority=low\|normal\|high` 优先级
|
|||
|
|
- `--queue` 触达上限时排队(最长 30s)
|
|||
|
|
|
|||
|
|
## 性能特征(重要)
|
|||
|
|
|
|||
|
|
养号场景每号约 30 操作 × 5s ≈ 2.5min。100 号串行 ≈ 4 小时。批量发布请用 `--humanize-mode=fast`。
|
|||
|
|
|
|||
|
|
## 详细文档
|
|||
|
|
|
|||
|
|
- [拟人化档位详解](references/humanization.md)
|
|||
|
|
- [指纹参数](references/fingerprint.md)
|
|||
|
|
- [代理配置](references/proxy.md)
|
|||
|
|
- [典型场景示例](references/examples.md)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step:Checkpoint**
|
|||
|
|
|
|||
|
|
## Task 20:references/ 子文档
|
|||
|
|
|
|||
|
|
> 创建 4 个 reference 文件作为 SKILL.md 的扩展。每个文件 50-100 行,按 SKILL.md 中链接对应的主题展开。
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Create: `packages/backend/skills/netabrowser-cli/references/humanization.md`
|
|||
|
|
- Create: `packages/backend/skills/netabrowser-cli/references/fingerprint.md`
|
|||
|
|
- Create: `packages/backend/skills/netabrowser-cli/references/proxy.md`
|
|||
|
|
- Create: `packages/backend/skills/netabrowser-cli/references/examples.md`
|
|||
|
|
|
|||
|
|
每个文件简明阐述对应主题,含示例命令。具体内容由实施者从 spec §6.1-6.6 + 实测脚本提炼。
|
|||
|
|
|
|||
|
|
- [ ] **Step:Checkpoint**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 21:Windows 打包集成
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Modify: `packages/backend/scripts/pkg-build.js`
|
|||
|
|
- Modify: `packages/backend/scripts/build-windows-installer.js`
|
|||
|
|
- Modify: `packages/backend/installer/setup.iss`
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: pkg-build.js 增加 netabrowser-cli 编译**
|
|||
|
|
|
|||
|
|
读 `pkg-build.js` 现有逻辑,在 backend.exe 编译之后增加:
|
|||
|
|
|
|||
|
|
```javascript
|
|||
|
|
// 编译 netabrowser-cli
|
|||
|
|
console.log('Building netabrowser-cli...');
|
|||
|
|
const cliRoot = path.resolve(__dirname, '../../netabrowser-cli');
|
|||
|
|
execSync('pnpm build', { cwd: cliRoot, stdio: 'inherit' });
|
|||
|
|
|
|||
|
|
// pkg 打包
|
|||
|
|
const pkgConfig = {
|
|||
|
|
name: 'netabrowser-cli',
|
|||
|
|
bin: path.join(cliRoot, 'dist', 'bin', 'main.js'),
|
|||
|
|
targets: ['node22-win-x64'],
|
|||
|
|
outputPath: path.join(__dirname, '../installer/dist/netabrowser-cli.exe'),
|
|||
|
|
};
|
|||
|
|
await pkg(pkgConfig);
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: setup.iss 增加 [Files] 段**
|
|||
|
|
|
|||
|
|
在现有 `[Files]` 后追加:
|
|||
|
|
|
|||
|
|
```iss
|
|||
|
|
Source: "dist/netabrowser-cli.exe"; DestDir: "{app}"; Flags: ignoreversion
|
|||
|
|
Source: "..\..\netabrowser-cli\chromium\win64\*"; DestDir: "{app}\chromium\win64"; Flags: ignoreversion recursesubdirs createallsubdirs
|
|||
|
|
Source: "..\..\backend\skills\netabrowser-cli\*"; DestDir: "{app}\skills\netabrowser-cli"; Flags: ignoreversion recursesubdirs createallsubdirs
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: build-windows-installer.js 加 chromium 复制步骤**
|
|||
|
|
|
|||
|
|
在调 ISCC 之前确保 `chromium/win64` 存在。
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 实跑一次安装包构建(dev 机器)**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
pnpm run build:windows-installer
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
预期:生成的 setup.exe 大小 ~250-400MB。装到测试 Windows,能找到 `C:\Program Files\Neta\netabrowser-cli.exe` 和 `chromium/win64/chrome.exe`。
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: Checkpoint**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 22:联调与冒烟(手工验收)
|
|||
|
|
|
|||
|
|
按 spec §9 验收清单逐条跑:
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 跑所有单测**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd packages/backend
|
|||
|
|
pnpm jest test/modules/netaclaw/browser-daemon/
|
|||
|
|
```
|
|||
|
|
预期:全 PASS
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 启动 backend dev 模式**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
NODE_ENV=development pnpm dev
|
|||
|
|
```
|
|||
|
|
预期:runtime-info.json 写入 `packages/backend/dist/runtime-info.json` 含 controlSecret(dev 模式下 secret 可能为空字符串,middleware 已开 dev 旁路)
|
|||
|
|
|
|||
|
|
> v2 注:dev 模式下 NETA_TRAY_SECRET 通常未设。middleware 已实现 dev 旁路:`NODE_ENV !== 'production' && expected === ''` → 仅校验 loopback、跳过 secret 校验。
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: cli list 应返回空**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
node packages/netabrowser-cli/dist/bin/main.js list --raw
|
|||
|
|
```
|
|||
|
|
预期:`[]`
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: 真代理出口 IP 验证**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
node packages/netabrowser-cli/dist/bin/main.js open https://httpbin.org/ip \
|
|||
|
|
--session=test \
|
|||
|
|
--proxy="http://e50b26:7ecdccfd@210.51.27.112:10000" \
|
|||
|
|
--headed
|
|||
|
|
```
|
|||
|
|
然后:
|
|||
|
|
```bash
|
|||
|
|
node packages/netabrowser-cli/dist/bin/main.js snapshot --session=test
|
|||
|
|
```
|
|||
|
|
看 url + 内容含 `"origin": "210.51.27.112"`
|
|||
|
|
|
|||
|
|
- [ ] **Step 5: creepjs 反检测 trust score**
|
|||
|
|
|
|||
|
|
打开 https://abrahamjuliot.github.io/creepjs/,肉眼记 trust score。预期 ≥ 70%。
|
|||
|
|
|
|||
|
|
- [ ] **Step 6: 拟人化档位延迟测量**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=full
|
|||
|
|
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=fast
|
|||
|
|
time node packages/netabrowser-cli/dist/bin/main.js click e1 --session=test --mode=off
|
|||
|
|
```
|
|||
|
|
预期:full ~2-5s, fast ~0.3-0.7s, off <0.2s
|
|||
|
|
|
|||
|
|
- [ ] **Step 7: HTTP 直接调,验证 auth**
|
|||
|
|
|
|||
|
|
**dev 旁路验证**:
|
|||
|
|
```bash
|
|||
|
|
# dev 模式(NODE_ENV=development,NETA_TRAY_SECRET 未设)
|
|||
|
|
curl http://localhost:8003/admin/browser-daemon/list
|
|||
|
|
# 200(dev 旁路允许 loopback)
|
|||
|
|
|
|||
|
|
# 非 loopback 仍 403
|
|||
|
|
curl --interface 8.8.8.8 http://localhost:8003/admin/browser-daemon/list
|
|||
|
|
# 403
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**prod 风格验证**:
|
|||
|
|
```bash
|
|||
|
|
# 杀掉 backend,重新启动 prod 模式
|
|||
|
|
NODE_ENV=production NETA_TRAY_SECRET=test-prod-secret pnpm start
|
|||
|
|
|
|||
|
|
# 无 secret → 401
|
|||
|
|
curl http://localhost:8003/admin/browser-daemon/list
|
|||
|
|
# 401
|
|||
|
|
|
|||
|
|
# 错误 secret → 401
|
|||
|
|
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: wrong"
|
|||
|
|
# 401
|
|||
|
|
|
|||
|
|
# 正确 secret → 200
|
|||
|
|
curl http://localhost:8003/admin/browser-daemon/list -H "x-neta-control-secret: test-prod-secret"
|
|||
|
|
# 200
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 8: 软上限验证**
|
|||
|
|
|
|||
|
|
设 `maxActiveSessions=2`,启动 3 个 session 看第 3 个返回 503。
|
|||
|
|
|
|||
|
|
- [ ] **Step 9: backend 重启 ref 失效**
|
|||
|
|
|
|||
|
|
ctrl+C 关 backend,重启 `pnpm dev`,调 `click e1 --session=test`。
|
|||
|
|
预期:报错(session 不存在或 ref 失效),需要先 `open` 再 `snapshot` 再 `click`。
|
|||
|
|
|
|||
|
|
- [ ] **Step 10: 安装包冒烟**
|
|||
|
|
|
|||
|
|
把 setup.exe 装到全新 Windows,启动托盘 → 后端启动 → cli 跑 step 4-5。
|
|||
|
|
|
|||
|
|
- [ ] **Step 11: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 22 完成`:所有验收标准通过。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Task 23:文档更新 + 路线图
|
|||
|
|
|
|||
|
|
**Files:**
|
|||
|
|
- Modify: `docs/superpowers/specs/2026-05-04-netabrowser-cli-s1-design.md`(变更日志)
|
|||
|
|
- Modify: `docs/superpowers/specs/2026-05-03-geo-master-roadmap.md`(如存在则更新)
|
|||
|
|
|
|||
|
|
- [ ] **Step 1: 更新 spec §13 变更日志**
|
|||
|
|
|
|||
|
|
加一行:"2026-XX-XX 实施完成,所有验收标准通过,准备进入 S2"。
|
|||
|
|
|
|||
|
|
- [ ] **Step 2: 更新 geo 路线图**
|
|||
|
|
|
|||
|
|
如有 geo master-roadmap,标注 netabrowser-cli S1 完成 → 解锁 geo BrowserAutomationService 迁移。
|
|||
|
|
|
|||
|
|
- [ ] **Step 3: 提示 user 走完整 git commit**
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd C:/Users/lixin/Desktop/RZYX_ZT/Neta-monorepo
|
|||
|
|
git status
|
|||
|
|
# 用户审核后统一 commit(如其要求)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- [ ] **Step 4: Checkpoint**
|
|||
|
|
|
|||
|
|
`Task 23 完成`:文档同步。等待 user 触发统一 commit。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 实施期间约束
|
|||
|
|
|
|||
|
|
- ❌ 不写 SQL 文件(如果有数据库需求,用 mcp mysql 工具直接 INSERT)
|
|||
|
|
- ❌ 每个 task **不单独 git commit**(用户要求全部完成 + 联调通过后统一提交)
|
|||
|
|
- ❌ 不修改现有 playwright-cli / patchwright-cli skill
|
|||
|
|
- ❌ 不在 controller 中加业务逻辑(只能是 thin wrapper,§5.3.1 强制)
|
|||
|
|
- ❌ HTTP API 必须强制 loopback + secret,不能例外
|
|||
|
|
- ❌ 不实现平台特定补丁(小红书/抖音/淘宝等)→ S2
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 实施完成后
|
|||
|
|
|
|||
|
|
- 调用 `superpowers:verification-before-completion` 逐条核对验收标准
|
|||
|
|
- 调用 `superpowers:requesting-code-review` 触发代码审查
|
|||
|
|
- 用户测试通过后统一 git commit
|
|||
|
|
- 更新 spec §13 变更日志
|
|||
|
|
- 进入下一个子项目:geo BrowserAutomationService 迁移到 BrowserDaemonService(S2)
|