592 lines
27 KiB
Markdown
592 lines
27 KiB
Markdown
|
# S1 基础设施层 设计文档
|
|||
|
|
|
|||
|
|
> Geo 模块第一个子项目：账号矩阵 / IP 池 / 指纹浏览器 / 菜单。
|
|||
|
|
> 上位文档：[`2026-05-03-geo-master-roadmap.md`](./2026-05-03-geo-master-roadmap.md)
|
|||
|
|
|
|||
|
|
| 元数据 | 值 |
|
|||
|
|
|---|---|
|
|||
|
|
| 子项目 | S1 基础设施层 |
|
|||
|
|
| 创建日期 | 2026-05-03 |
|
|||
|
|
| 估期 | ~9 天 |
|
|||
|
|
| 状态 | 设计完成，待 user 复核 → writing-plans |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 1. 目标
|
|||
|
|
|
|||
|
|
为 Geo 模块后续所有子项目（S2/S3/S4）提供**账号—IP—指纹浏览器**三位一体的基础设施。S1 完成后，用户能在 Neta 控制台：
|
|||
|
|
|
|||
|
|
1. 在 GEO 一级菜单下看到三个子页面
|
|||
|
|
2. 创建一个 IP（本地或第三方）
|
|||
|
|
3. 创建一个指纹浏览器 profile
|
|||
|
|
4. 创建一个社媒账号，自动绑定 IP + Profile（强 1:1:1）
|
|||
|
|
5. 点击「启动登录」打开真实浏览器，扫码/输账密/输短信完成登录
|
|||
|
|
6. Cookie 自动抱回并加密存储
|
|||
|
|
7. 删除账号时安全释放 IP 和 Profile
|
|||
|
|
|
|||
|
|
S1 **不涉及**任何平台特定行为（发布、评论、监测）。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 2. 关键决策（brainstorming 阶段已确认）
|
|||
|
|
|
|||
|
|
| 决策点 | 决策 |
|
|||
|
|
|---|---|
|
|||
|
|
| 子项目拆分 | 4 个子项目（S1/S2/S3/S4），先做 S1 |
|
|||
|
|
| IP 池 | 抽象 `IProxyProvider` + 内置 `LocalProvider` + 占位 `TianqiProvider` |
|
|||
|
|
| 浏览器（两层正交） | ① 浏览器进程层 `IBrowserProvider`：S1 主实现 `PlainChromiumProvider`（playwright-cli 直接启 Chromium）+ 占位 `BitBrowser` `AntBrowser` `AdsPower`。② 自动化层 `BrowserAutomationService`：统一用 `playwright-cli -s={session}` 做 cookie/state/click/type，不关心底层是哪种浏览器 |
|
|||
|
|
| 绑定关系 | account ↔ proxy_ip ↔ browser_profile **严格 1:1:1**，IP/Profile 上 `bindAccountId` 唯一索引 |
|
|||
|
|
| 登录方式 | `playwright-cli -s={session} open {url} --persistent --headed` 启动有头浏览器 + 用户自己登录 + `BrowserAutomationService.getCookies()` 自动抱回 |
|
|||
|
|
| 实现风格 | 分层 Provider 抽象 + account service 单点编排（与 NetaClaw `llm_providers/` 对齐） |
|
|||
|
|
| 数据库 | 不写 SQL 文件；Entity + TypeORM `synchronize` 自动建表；菜单 seed 用 `mcp__mysql__execute` |
|
|||
|
|
| 加密 | geo 模块自建 `GeoEncryptService`（复用同一 AES-256-GCM 算法和 `SKILL_SECRET_KEY`/`APP_SECRET` 密钥派生逻辑，但接口签名为 `encrypt(plain: string): string` / `decrypt(cipher: string): string`，适配 cookie/密码等字符串场景。不直接调用 `SkillSecretService`——后者签名为 `encrypt(obj: Record<string,string>)` 且语义是 skill scoped secrets） |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 3. 与 Neta 现有架构契合度审计
|
|||
|
|
|
|||
|
|
| Neta 约定 | S1 实施 |
|
|||
|
|
|---|---|
|
|||
|
|
| 模块根目录 `src/modules/{name}/` | ✅ `src/modules/geo/` |
|
|||
|
|
| `entity/` 文件下划线 + BaseEntity + 字段驼峰 | ✅ |
|
|||
|
|
| `controller/admin/` + `@CoolController` 自动 CRUD | ✅ |
|
|||
|
|
| `service/` 业务编排 | ✅ |
|
|||
|
|
| `entities.ts` 注册 | ✅ 自动生成（`cool entity` 命令扫描 `modules/*/entity` 通配符，无需手动修改） |
|
|||
|
|
| 前端模块 `config.ts` 导出 ModuleConfig | ✅ |
|
|||
|
|
| 加密复用 AES-256-GCM 算法和密钥派生 | ✅ 自建 `GeoEncryptService`（复用算法+密钥，接口适配字符串场景） |
|
|||
|
|
| 菜单走 base_sys_menu | ✅ |
|
|||
|
|
| TypeORM `synchronize: true` 自动建表 | ✅（dev 环境，prod 禁用） |
|
|||
|
|
| 多租户 tenantId 自动注入 | ✅ |
|
|||
|
|
| 不在 S1：gateway/skill/runtime（属过度设计） | ✅ |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 4. 架构与目录
|
|||
|
|
|
|||
|
|
### 4.1 后端目录
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
packages/backend/src/
|
|||
|
|
└── modules/geo/ # entities.ts 由 `cool entity` 自动生成，无需手动修改
|
|||
|
|
├── config.ts
|
|||
|
|
├── entity/
|
|||
|
|
│ ├── account.ts # geo_account
|
|||
|
|
│ ├── proxy_ip.ts # geo_proxy_ip
|
|||
|
|
│ └── browser_profile.ts # geo_browser_profile
|
|||
|
|
├── controller/admin/
|
|||
|
|
│ ├── account.ts # @CoolController + 自定义 launch / captureCookies
|
|||
|
|
│ ├── proxy_ip.ts # @CoolController + 自定义 healthCheck / batchImport
|
|||
|
|
│ └── browser_profile.ts # @CoolController + 自定义 open / close
|
|||
|
|
├── service/
|
|||
|
|
│ ├── account.ts # ★ 单点编排：bindResources / launch / captureCookies / rebindIp
|
|||
|
|
│ ├── proxy_ip.ts # CRUD 包装 + provider 调度
|
|||
|
|
│ ├── browser_profile.ts # CRUD 包装 + provider 调度
|
|||
|
|
│ ├── browser_automation.ts # ★ 统一自动化层：playwright-cli 包装（cookie/state/click/type）
|
|||
|
|
│ └── encrypt.ts # GeoEncryptService
|
|||
|
|
└── provider/ # geo 模块内部约定（非 NetaClaw plugins/）
|
|||
|
|
# 原因：geo provider 不需要动态加载，且 plugins/ 在 Neta 语义里
|
|||
|
|
# 特指 LLM 适配层（netaclaw/plugins/llm_providers/）
|
|||
|
|
├── proxy/
|
|||
|
|
│ ├── interface.ts # IProxyProvider
|
|||
|
|
│ ├── local.ts # LocalProxyProvider
|
|||
|
|
│ └── tianqi.ts # 占位 TianqiProxyProvider（等文档）
|
|||
|
|
└── browser/
|
|||
|
|
├── interface.ts # IBrowserProvider（仅进程生命周期）
|
|||
|
|
├── plain_chromium.ts # ★ S1 主实现：playwright-cli + Chromium
|
|||
|
|
├── bitbrowser.stub.ts # 占位（启动 BitBrowser，playwright-cli attach）
|
|||
|
|
├── ant_browser.stub.ts # 占位（等用户提供源码）
|
|||
|
|
└── adspower.stub.ts # 占位
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 4.2 前端目录
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
packages/frontend/src/modules/geo/
|
|||
|
|
├── config.ts # ★ 必须 export ModuleConfig
|
|||
|
|
└── views/ # service 代理由 Cool Admin 根据后端 controller 文件名自动生成
|
|||
|
|
# 如 service.geo.account.page() / service.geo.proxy_ip.healthCheck()
|
|||
|
|
# 无需手动创建 service/ 目录
|
|||
|
|
├── dashboard.vue # 占位（S4 才正式做内容）
|
|||
|
|
├── accounts.vue # 账号矩阵主页
|
|||
|
|
├── proxies.vue # IP 池
|
|||
|
|
└── browser-profiles.vue # 指纹浏览器
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 4.3 菜单 seed（运行时通过 MCP mysql 执行 INSERT）
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
🌍 GEO（一级目录，icon=icon-trend，order=50）
|
|||
|
|
├── 账号矩阵 viewPath=modules/geo/views/accounts.vue perms=geo:account:list
|
|||
|
|
├── IP 池 viewPath=modules/geo/views/proxies.vue perms=geo:proxy:list
|
|||
|
|
└── 指纹浏览器 viewPath=modules/geo/views/browser-profiles.vue perms=geo:profile:list
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
按钮权限：`geo:account:add` / `update` / `delete` / `launch` / `captureCookies` / `rebindIp`；`geo:proxy:add` / `update` / `delete` / `healthCheck` / `batchImport`；`geo:profile:add` / `update` / `delete` / `open` / `close`。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 5. 数据模型
|
|||
|
|
|
|||
|
|
### 5.1 geo_account
|
|||
|
|
|
|||
|
|
| 字段 | 类型 | 说明 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `id` | bigint PK | BaseEntity |
|
|||
|
|
| `tenantId` | bigint Index | BaseEntity 自动注入 |
|
|||
|
|
| `name` | varchar(64) | 备注名/昵称 |
|
|||
|
|
| `platform` | varchar(32) Index | xiaohongshu / douyin / weibo / zhihu / wechat |
|
|||
|
|
| `loginAccount` | varchar(128) | 登录手机号/账号 |
|
|||
|
|
| `cookies` | text | **加密** JSON |
|
|||
|
|
| `cookieCapturedAt` | datetime NULL | |
|
|||
|
|
| `loginStatus` | varchar(16) | never / logged_in / expired |
|
|||
|
|
| `status` | varchar(16) | draft / active / risky / banned / deleted |
|
|||
|
|
| `proxyId` | bigint UQ Index | → geo_proxy_ip.id |
|
|||
|
|
| `browserProfileId` | bigint UQ Index | → geo_browser_profile.id |
|
|||
|
|
| `personaId` | bigint NULL | S2 用，S1 占位 |
|
|||
|
|
| `agentConfigId` | bigint NULL | S3 用 |
|
|||
|
|
| `lastActiveAt` | datetime NULL | |
|
|||
|
|
| `extra` | json | 平台特定字段 |
|
|||
|
|
| `createTime` `updateTime` | datetime | BaseEntity |
|
|||
|
|
|
|||
|
|
**约束**：(`platform`, `loginAccount`) 联合唯一；`proxyId` 唯一；`browserProfileId` 唯一。
|
|||
|
|
|
|||
|
|
### 5.2 geo_proxy_ip
|
|||
|
|
|
|||
|
|
| 字段 | 类型 | 说明 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `id` | bigint PK | |
|
|||
|
|
| `tenantId` | bigint Index | |
|
|||
|
|
| `name` | varchar(64) | |
|
|||
|
|
| `provider` | varchar(32) | local / tianqi |
|
|||
|
|
| `mode` | varchar(16) | local / third_party |
|
|||
|
|
| `host` | varchar(128) NULL | 第三方才有 |
|
|||
|
|
| `port` | int NULL | |
|
|||
|
|
| `protocol` | varchar(8) | http / socks5 |
|
|||
|
|
| `username` | varchar(128) NULL | **加密** |
|
|||
|
|
| `password` | varchar(255) NULL | **加密** |
|
|||
|
|
| `region` | varchar(64) | 城市/省份 |
|
|||
|
|
| `isp` | varchar(32) | 联通/电信/移动 |
|
|||
|
|
| `externalId` | varchar(128) NULL | provider 侧 ID |
|
|||
|
|
| `bindAccountId` | bigint UQ NULL Index | 反向唯一 |
|
|||
|
|
| `status` | varchar(16) | active / expired / error / unbound |
|
|||
|
|
| `latencyMs` | int NULL | |
|
|||
|
|
| `lastCheckAt` | datetime NULL | |
|
|||
|
|
| `expiresAt` | datetime NULL | |
|
|||
|
|
| `extra` | json | |
|
|||
|
|
|
|||
|
|
### 5.3 geo_browser_profile
|
|||
|
|
|
|||
|
|
| 字段 | 类型 | 说明 |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `id` | bigint PK | |
|
|||
|
|
| `tenantId` | bigint Index | |
|
|||
|
|
| `name` | varchar(64) | |
|
|||
|
|
| `provider` | varchar(32) | plain_chromium / bitbrowser / ant_browser / adspower |
|
|||
|
|
| `sessionName` | varchar(128) Index | playwright-cli 的 `-s=xxx` 命名 session（唯一标识） |
|
|||
|
|
| `accountId` | bigint UQ NULL Index | 反向唯一 |
|
|||
|
|
| `profileDir` | varchar(512) NULL | `--profile=/path` 持久化目录 |
|
|||
|
|
| `configPath` | varchar(512) NULL | `--config=xxx.json` 代理配置文件路径 |
|
|||
|
|
| `userAgent` | varchar(512) | |
|
|||
|
|
| `osPlatform` | varchar(32) | windows / mac / linux |
|
|||
|
|
| `timezone` | varchar(32) | |
|
|||
|
|
| `language` | varchar(16) | |
|
|||
|
|
| `screenW` | int | |
|
|||
|
|
| `screenH` | int | |
|
|||
|
|
| `fingerprint` | json | 深度指纹配置（S1 不填，等 ant-browser） |
|
|||
|
|
| `lastOpenAt` | datetime NULL | |
|
|||
|
|
| `status` | varchar(16) | created / running / closed / error / deleted |
|
|||
|
|
| `extra` | json | |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 6. Provider 接口契约
|
|||
|
|
|
|||
|
|
### 6.1 IProxyProvider
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// provider/proxy/interface.ts
|
|||
|
|
export interface AcquireOpts {
|
|||
|
|
region?: string;
|
|||
|
|
isp?: string;
|
|||
|
|
duration?: 'fixed' | 'rotating';
|
|||
|
|
// 第三方扩展字段
|
|||
|
|
extra?: Record<string, any>;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface ProxyInfo {
|
|||
|
|
externalId: string;
|
|||
|
|
mode: 'local' | 'third_party';
|
|||
|
|
host?: string;
|
|||
|
|
port?: number;
|
|||
|
|
protocol: 'http' | 'socks5';
|
|||
|
|
username?: string;
|
|||
|
|
password?: string;
|
|||
|
|
region?: string;
|
|||
|
|
isp?: string;
|
|||
|
|
expiresAt?: Date;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface IProxyProvider {
|
|||
|
|
readonly name: string;
|
|||
|
|
acquire(opts: AcquireOpts): Promise<ProxyInfo>;
|
|||
|
|
release(externalId: string): Promise<void>;
|
|||
|
|
healthCheck(p: ProxyInfo): Promise<{ ok: boolean; latencyMs: number }>;
|
|||
|
|
list?(): Promise<ProxyInfo[]>;
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 6.2 IBrowserProvider（仅浏览器进程生命周期，不含自动化）
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// provider/browser/interface.ts
|
|||
|
|
import type { ProxyInfo } from '../proxy/interface';
|
|||
|
|
|
|||
|
|
export interface CreateOpts {
|
|||
|
|
name: string;
|
|||
|
|
sessionName: string;
|
|||
|
|
userAgent?: string;
|
|||
|
|
osPlatform?: 'windows' | 'mac' | 'linux';
|
|||
|
|
timezone?: string;
|
|||
|
|
language?: string;
|
|||
|
|
screenW?: number;
|
|||
|
|
screenH?: number;
|
|||
|
|
fingerprint?: Record<string, any>;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface ProfileInfo {
|
|||
|
|
sessionName: string;
|
|||
|
|
profileDir?: string;
|
|||
|
|
configPath?: string;
|
|||
|
|
userAgent: string;
|
|||
|
|
osPlatform: string;
|
|||
|
|
timezone: string;
|
|||
|
|
language: string;
|
|||
|
|
screenW: number;
|
|||
|
|
screenH: number;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
export interface IBrowserProvider {
|
|||
|
|
readonly name: string;
|
|||
|
|
/** 创建 profile（写配置文件、注册指纹环境，不启动进程） */
|
|||
|
|
create(opts: CreateOpts): Promise<ProfileInfo>;
|
|||
|
|
/** 删除 profile 配置 */
|
|||
|
|
delete(profile: ProfileInfo): Promise<void>;
|
|||
|
|
/** 把代理写入 profile 配置（attachProxy 与 open 解耦） */
|
|||
|
|
attachProxy(profile: ProfileInfo, proxy: ProxyInfo): Promise<void>;
|
|||
|
|
/** 启动浏览器进程并打开 URL（最终 BrowserAutomationService 通过 sessionName 操作） */
|
|||
|
|
open(profile: ProfileInfo, url?: string): Promise<void>;
|
|||
|
|
/** 关闭浏览器进程 */
|
|||
|
|
close(profile: ProfileInfo): Promise<void>;
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 6.3 Provider 实现职责
|
|||
|
|
|
|||
|
|
| 实现 | 关键行为 |
|
|||
|
|
|---|---|
|
|||
|
|
| `LocalProxyProvider` | acquire 直接返回 `mode='local'` 标记的 ProxyInfo（不连出站代理）；healthCheck 走宿主机 https://www.baidu.com |
|
|||
|
|
| `TianqiProxyProvider`（占位） | acquire/release 抛 `NotImplemented` 但不阻塞模块加载；healthCheck 同上；文件头部注释列出对接 TODO |
|
|||
|
|
| **`PlainChromiumProvider`**（★ S1 主实现） | create → 在 `dataDir/geo/profiles/` 下生成 `geo-{accountId}` profile 目录 + 写一个 `geo-proxy-{accountId}.json` 代理配置；attachProxy → 重写代理 JSON；open → exec `playwright-cli -s={session} open {url} --persistent --profile={profileDir} --headed --config={configPath}`；close → exec `playwright-cli -s={session} close`；delete → `playwright-cli -s={session} delete-data` + 删除配置文件 |
|
|||
|
|
| `BitBrowserStub` | 占位。未来实现：调 BitBrowser Local API 启动 profile 拿到 CDP endpoint，playwright-cli 通过 attach 模式连进去做自动化 |
|
|||
|
|
| `AntBrowserStub` | 占位。等用户提供源码后实现 |
|
|||
|
|
| `AdsPowerStub` | 占位。仅声明 name + 抛 NotImplemented |
|
|||
|
|
|
|||
|
|
### 6.4 BrowserAutomationService（统一自动化层，所有 Provider 共用）
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
// service/browser_automation.ts
|
|||
|
|
@Provide()
|
|||
|
|
export class BrowserAutomationService {
|
|||
|
|
/** 列出指定域名的 cookie。底层 exec：playwright-cli -s={session} cookie-list --domain={domain} --raw */
|
|||
|
|
async getCookies(sessionName: string, domain?: string): Promise<Cookie[]>;
|
|||
|
|
|
|||
|
|
/** 保存完整登录态到文件。底层 exec：playwright-cli -s={session} state-save {path} */
|
|||
|
|
async saveState(sessionName: string, filePath: string): Promise<void>;
|
|||
|
|
|
|||
|
|
/** 恢复登录态。底层 exec：playwright-cli -s={session} state-load {path} */
|
|||
|
|
async loadState(sessionName: string, filePath: string): Promise<void>;
|
|||
|
|
|
|||
|
|
/** S3 后续会增加 click / type / snapshot 等自动化方法 */
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**关键设计**：`BrowserAutomationService` 与 `IBrowserProvider` 完全解耦。无论 provider 启动的是普通 Chromium 还是 BitBrowser/ant-browser，只要进程存在且 `playwright-cli -s={session}` 能命中（或 attach 进去），自动化层无差别工作。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 7. 核心数据流
|
|||
|
|
|
|||
|
|
### 7.1 创建账号（同时分配 IP + Profile）
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
前端 accounts.vue → POST /admin/geo/account/add
|
|||
|
|
↓
|
|||
|
|
service/account.ts.add(dto):
|
|||
|
|
await dataSource.transaction(async manager => {
|
|||
|
|
// 1. 申请 IP
|
|||
|
|
const proxyDto = dto.ipMode === 'local'
|
|||
|
|
? { provider: 'local', mode: 'local' }
|
|||
|
|
: { provider: 'tianqi', mode: 'third_party', region: dto.region };
|
|||
|
|
const ipInfo = await proxyService.acquire(proxyDto);
|
|||
|
|
const ipEntity = await proxyService.persist(manager, ipInfo);
|
|||
|
|
|
|||
|
|
// 2. 申请 Profile
|
|||
|
|
const profileInfo = await browserService.create({
|
|||
|
|
provider: dto.browserProvider ?? 'bitbrowser',
|
|||
|
|
...dto.fingerprint
|
|||
|
|
});
|
|||
|
|
const profileEntity = await browserService.persist(manager, profileInfo);
|
|||
|
|
|
|||
|
|
// 3. Provider 侧挂代理
|
|||
|
|
await browserService.attachProxy(profileEntity, ipEntity);
|
|||
|
|
|
|||
|
|
// 4. 创建 account
|
|||
|
|
const account = manager.create(GeoAccount, {
|
|||
|
|
...dto, proxyId: ipEntity.id, browserProfileId: profileEntity.id
|
|||
|
|
});
|
|||
|
|
return manager.save(account);
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
// 失败补偿：transaction 回滚 + Provider 端
|
|||
|
|
catch (e) {
|
|||
|
|
if (ipExternal) await proxyProvider.release(ipExternal);
|
|||
|
|
if (profileExternal) await browserProvider.delete(profileExternal);
|
|||
|
|
throw e;
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 7.2 启动浏览器环境
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
前端 [启动登录] → POST /admin/geo/account/launch { id, url? }
|
|||
|
|
↓
|
|||
|
|
service/account.ts.launch(id, url?):
|
|||
|
|
const account = repo.findOneOrFail(id);
|
|||
|
|
const profile = profileRepo.findOneOrFail(account.browserProfileId);
|
|||
|
|
const targetUrl = url || PLATFORM_URLS[account.platform]; // 如 https://www.xiaohongshu.com
|
|||
|
|
await browserProvider.open(profile.sessionName, targetUrl);
|
|||
|
|
// playwright-cli 会弹出有头浏览器窗口，用户在里面登录
|
|||
|
|
await profileRepo.update(profile.id, { status: 'running', lastOpenAt: new Date() });
|
|||
|
|
return { sessionName: profile.sessionName };
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 7.3 抓 Cookie
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
前端轮询/手动触发 → POST /admin/geo/account/captureCookies { id, domains? }
|
|||
|
|
↓
|
|||
|
|
service/account.ts.captureCookies(id, domains?):
|
|||
|
|
const account = ...findOneOrFail(id);
|
|||
|
|
const profile = ...findOneOrFail(account.browserProfileId);
|
|||
|
|
if (profile.status !== 'running') throw new Error('Browser not running, call launch first');
|
|||
|
|
|
|||
|
|
// 自动化层走 BrowserAutomationService（与底层是哪种浏览器无关）
|
|||
|
|
const domain = domains?.[0];
|
|||
|
|
const cookies = await browserAutomationService.getCookies(profile.sessionName, domain);
|
|||
|
|
if (cookies.length === 0) {
|
|||
|
|
return { captured: 0 };
|
|||
|
|
}
|
|||
|
|
const encrypted = geoEncryptService.encrypt(JSON.stringify(cookies));
|
|||
|
|
await accountRepo.update(id, {
|
|||
|
|
cookies: encrypted,
|
|||
|
|
cookieCapturedAt: new Date(),
|
|||
|
|
loginStatus: 'logged_in',
|
|||
|
|
});
|
|||
|
|
|
|||
|
|
// 同时保存完整登录态（cookie + localStorage）到文件，方便后续 loadState 复活会话
|
|||
|
|
const statePath = `${dataDir}/geo/states/geo-state-${id}.json`;
|
|||
|
|
await browserAutomationService.saveState(profile.sessionName, statePath);
|
|||
|
|
|
|||
|
|
return { captured: cookies.length };
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 7.4 IP 健康检查（Neta task 模块 cron）
|
|||
|
|
|
|||
|
|
Neta 的 task 模块通过 `task_info` 表 + `TaskLocalService`（基于 `cron` npm 包）调度。注册方式：在 `task_info` 表中 INSERT 一条记录，`service` 字段指向要调用的 service 方法路径。
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
注册（通过 mcp__mysql__execute INSERT 到 task_info 表）：
|
|||
|
|
name: 'GEO IP 健康检查'
|
|||
|
|
service: 'geo.proxy_ip.healthCheckAll'
|
|||
|
|
type: 0 # 0=cron 表达式
|
|||
|
|
cron: '0 0 */6 * * *' # 每 6 小时
|
|||
|
|
status: 1 # 启用
|
|||
|
|
|
|||
|
|
service/proxy_ip.ts:
|
|||
|
|
async healthCheckAll() {
|
|||
|
|
const ips = await this.proxyIpEntity.find({ where: { status: 'active' } });
|
|||
|
|
for (const ip of ips) {
|
|||
|
|
const provider = this.getProvider(ip.provider);
|
|||
|
|
const result = await provider.healthCheck(this.toProxyInfo(ip));
|
|||
|
|
await this.proxyIpEntity.update(ip.id, {
|
|||
|
|
latencyMs: result.latencyMs,
|
|||
|
|
lastCheckAt: new Date(),
|
|||
|
|
status: result.ok ? 'active' : 'error',
|
|||
|
|
});
|
|||
|
|
if (!result.ok) {
|
|||
|
|
this.logger.warn(`[GEO] IP ${ip.id} (${ip.host}) health check failed`);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
// S1 仅 logger.warn，不入告警表（S4 才有 geo_alert）
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 7.5 解绑/换 IP
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
service/account.ts.rebindIp(accountId, newIpId):
|
|||
|
|
await dataSource.transaction(async manager => {
|
|||
|
|
const account = await manager.findOneOrFail(GeoAccount, accountId);
|
|||
|
|
const oldIp = await manager.findOne(GeoProxyIp, account.proxyId);
|
|||
|
|
const newIp = await manager.findOneOrFail(GeoProxyIp, newIpId);
|
|||
|
|
if (newIp.bindAccountId) throw new Error('IP 已被其他账号绑定');
|
|||
|
|
|
|||
|
|
if (oldIp) {
|
|||
|
|
await manager.update(GeoProxyIp, oldIp.id, { bindAccountId: null, status: 'unbound' });
|
|||
|
|
}
|
|||
|
|
await manager.update(GeoProxyIp, newIp.id, { bindAccountId: accountId, status: 'active' });
|
|||
|
|
await manager.update(GeoAccount, accountId, { proxyId: newIp.id });
|
|||
|
|
|
|||
|
|
const profile = await manager.findOneOrFail(GeoBrowserProfile, account.browserProfileId);
|
|||
|
|
await browserProvider.attachProxy(profile.externalProfileId, toProxyInfo(newIp));
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 7.6 删除账号
|
|||
|
|
|
|||
|
|
Neta 现有模块不使用 TypeORM `softRemove`（BaseEntity 没有 `@DeleteDateColumn`）。geo 模块采用**逻辑删除**：account 和 profile 设 `status='deleted'`，与 Neta 其他模块风格一致。
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
service/account.ts.deleteAccount(id):
|
|||
|
|
await dataSource.transaction(async manager => {
|
|||
|
|
const account = await manager.findOneOrFail(GeoAccount, { where: { id } });
|
|||
|
|
const ip = account.proxyId
|
|||
|
|
? await manager.findOne(GeoProxyIp, { where: { id: account.proxyId } })
|
|||
|
|
: null;
|
|||
|
|
const profile = account.browserProfileId
|
|||
|
|
? await manager.findOne(GeoBrowserProfile, { where: { id: account.browserProfileId } })
|
|||
|
|
: null;
|
|||
|
|
|
|||
|
|
if (profile) {
|
|||
|
|
await browserProvider.delete(profile.externalProfileId).catch(() => {/* 容忍外部失败 */});
|
|||
|
|
await manager.update(GeoBrowserProfile, profile.id, { status: 'deleted', accountId: null });
|
|||
|
|
}
|
|||
|
|
if (ip) {
|
|||
|
|
await manager.update(GeoProxyIp, ip.id, { bindAccountId: null, status: 'unbound' });
|
|||
|
|
}
|
|||
|
|
await manager.update(GeoAccount, id, { status: 'deleted', proxyId: null, browserProfileId: null });
|
|||
|
|
});
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 8. 错误处理
|
|||
|
|
|
|||
|
|
| 故障 | 处理 |
|
|||
|
|
|---|---|
|
|||
|
|
| playwright-cli 未安装 | `PlaywrightCliProvider` 启动时检测 `playwright-cli --version`，失败抛 `BrowserProviderUnavailable`，前端弹窗提示安装 |
|
|||
|
|
| 天启 API 超时 | 重试 3 次（指数退避），仍失败 → `ProxyAcquireFailed`，前端提示切 local 或重试 |
|
|||
|
|
| Cookie 抓取空 | 不写库，返回 `captured: 0`，前端提示用户先在打开的浏览器里完成登录 |
|
|||
|
|
| 事务中途失败 | DB 自动回滚 + Provider 端补偿 release/delete |
|
|||
|
|
| 并发竞争 IP | UQ 索引兜底；service 层先 `lockMode: 'pessimistic_write'` 锁住 IP |
|
|||
|
|
| 浏览器进程残留 | `close` 失败时 logger.error 但不阻塞业务，运维侧定期清理 |
|
|||
|
|
| 加密密钥丢失 | `GeoEncryptService` 复用 `SKILL_SECRET_KEY`/`APP_SECRET` 环境变量派生密钥，丢失时抛 `EncryptionKeyMissing` |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 9. 测试策略
|
|||
|
|
|
|||
|
|
| 层 | 方法 |
|
|||
|
|
|---|---|
|
|||
|
|
| **Provider 单元** | LocalProxyProvider / BitBrowserProvider 走 mock fetch；TianqiProvider 验证抛 NotImplemented |
|
|||
|
|
| **Service 单元** | jest + sqlite，验证编排逻辑、事务回滚、补偿动作触发 |
|
|||
|
|
| **Controller 集成** | jest + supertest，跑 add → launch → captureCookies 一遍 |
|
|||
|
|
| **手工冒烟** | 真机 BitBrowser + 真小红书账号：创建 → 启动 → 登录 → 抓 cookie → loginStatus 验证 |
|
|||
|
|
|
|||
|
|
测试不在 S1 范围放真账号 token，所有真实凭据只在手工冒烟阶段使用。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 10. 验收标准
|
|||
|
|
|
|||
|
|
1. ✅ 后端 `pnpm dev` 启动，TypeORM 自动建好 `geo_account` `geo_proxy_ip` `geo_browser_profile` 三张表（可用 `mcp__mysql__list_tables` 验证）
|
|||
|
|
2. ✅ Cool Admin 自动 CRUD 接口可用：`/admin/geo/{account,proxy_ip,browser_profile}/{add,delete,update,info,list,page}` 全部返回 200
|
|||
|
|
3. ✅ 自定义接口可用：`/admin/geo/account/launch` `/captureCookies` `/rebindIp`、`/admin/geo/proxy_ip/healthCheck` `/batchImport`、`/admin/geo/browser_profile/open` `/close`
|
|||
|
|
4. ✅ `mcp__mysql__query` 能查到 `base_sys_menu` 中 `🌍 GEO` 顶级菜单 + 3 个子菜单 + 全部按钮权限
|
|||
|
|
5. ✅ 前端登录后菜单出现 `🌍 GEO` + 3 个子项；点击进入对应页面
|
|||
|
|
6. ✅ 在前端能完整跑通：
|
|||
|
|
- 创建一个 Local IP，能跑 healthCheck 看到延迟数字
|
|||
|
|
- 创建一个 playwright-cli profile，能 open 弹出有头浏览器窗口
|
|||
|
|
- 创建一个账号（platform=xiaohongshu），自动绑定 IP+Profile
|
|||
|
|
- 点击「启动登录」弹出有头浏览器，登录小红书后点「抓取 Cookie」，loginStatus 变 logged_in，cookies 字段非空且加密
|
|||
|
|
7. ✅ 删除账号：DB 中 IP 置 unbound 但记录保留；Profile 状态置 deleted；account 状态置 deleted（逻辑删除，非 softRemove）
|
|||
|
|
8. ✅ 强绑定约束：尝试给已绑账号的 IP 重新绑给别人，DB 报唯一约束错（前端友好提示）
|
|||
|
|
9. ✅ playwright-cli 未安装时，前端 open 操作返回 `BrowserProviderUnavailable` 错误并弹窗
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 11. 交付物清单
|
|||
|
|
|
|||
|
|
| 类别 | 文件 |
|
|||
|
|
|---|---|
|
|||
|
|
| **后端 entity** | `entity/account.ts` `entity/proxy_ip.ts` `entity/browser_profile.ts` |
|
|||
|
|
| **后端 controller** | `controller/admin/account.ts` `proxy_ip.ts` `browser_profile.ts` |
|
|||
|
|
| **后端 service** | `service/account.ts` `service/proxy_ip.ts` `service/browser_profile.ts` `service/browser_automation.ts`（统一自动化层） `service/encrypt.ts`（GeoEncryptService） |
|
|||
|
|
| **Provider** | `provider/proxy/{interface,local,tianqi}.ts` `provider/browser/{interface,plain_chromium,bitbrowser.stub,ant_browser.stub,adspower.stub}.ts` |
|
|||
|
|
| **配置** | `modules/geo/config.ts`（`entities.ts` 由 `cool entity` 自动生成，无需手动改） |
|
|||
|
|
| **菜单** | `mcp__mysql__execute` 注入 `base_sys_menu`（4 条 menu + 12+ 条 perms） |
|
|||
|
|
| **前端** | `modules/geo/{config.ts, views/{accounts,proxies,browser-profiles,dashboard}.vue, components/}` |
|
|||
|
|
| **测试** | `test/modules/geo/**` 单元+集成 |
|
|||
|
|
| **文档** | 本 spec + 后续 plan |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 12. 风险与依赖
|
|||
|
|
|
|||
|
|
| 风险/依赖 | 缓解 |
|
|||
|
|
|---|---|
|
|||
|
|
| playwright-cli 未安装 | Provider 启动时检测 + 前端友好提示 + 文档说明安装步骤（`npm i -g @playwright/cli@latest`） |
|
|||
|
|
| 天启 HTTP 文档迟到 | TianqiProvider 占位实现可走通整体流程，文档到位后只改一文件 |
|
|||
|
|
| `synchronize: true` 在 prod 危险 | Neta 现有约定 prod 禁用，S1 不改前提 |
|
|||
|
|
| Cookie 抓取目标域不全 | captureCookies 接受 `domains?: string[]` 参数 |
|
|||
|
|
| Playwright 与代理兼容性 | playwright-cli `--config` 支持 proxy 配置；S1 验证 local + http proxy 两种场景 |
|
|||
|
|
| 测试环境没有 playwright-cli | Provider 抽象允许 mock child_process.exec；CI 用 mock |
|
|||
|
|
| 深度指纹伪装 | S1 不做（playwright-cli 不支持 canvas/webgl 伪装）；等 ant-browser 源码到位后新增 AntBrowserProvider |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 13. 范围红线（不在 S1）
|
|||
|
|
|
|||
|
|
- ❌ 平台特定登录流程（QR 轮询/账密表单/短信） → S3
|
|||
|
|
- ❌ 浏览/发布/评论/指标采集 → S3
|
|||
|
|
- ❌ 关键词、内容、Schema → S2
|
|||
|
|
- ❌ AI 引用监测 → S4
|
|||
|
|
- ❌ 多模态/数字人/爆款拆解 → S5（暂不规划）
|
|||
|
|
- ❌ Skill 包对外暴露给 NetaClaw Agent → S3 起做
|
|||
|
|
- ❌ WebSocket 实时启动日志 → S3 评估是否需要
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 14. 工作流
|
|||
|
|
|
|||
|
|
完成本 spec 后：
|
|||
|
|
|
|||
|
|
1. user 复核本文档
|
|||
|
|
2. 调用 `superpowers:writing-plans` 生成 `docs/superpowers/plans/2026-05-03-geo-s1-infrastructure-plan.md`
|
|||
|
|
3. user 复核 plan
|
|||
|
|
4. `superpowers:executing-plans` + TDD 实施
|
|||
|
|
5. `superpowers:verification-before-completion` 逐条核对验收标准
|
|||
|
|
6. `superpowers:requesting-code-review`
|
|||
|
|
7. 更新主路线图：S1 状态 → 已完成；启动 S2 brainstorming
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 15. 变更日志
|
|||
|
|
|
|||
|
|
| 日期 | 变更 |
|
|||
|
|
|---|---|
|
|||
|
|
| 2026-05-03 | 初稿 |
|
|||
|
|
| 2026-05-03 | 架构审查修复 8 项：①entities.ts 自动生成不手动改 ②自建 GeoEncryptService 替代直接调用 SkillSecretService ③定时任务改用 task_info 表注册 ④provider/ 目录加说明 ⑤删除前端 service/ 目录 ⑥逻辑删除替代 softRemove ⑦BitBrowser 端口从 config/env 读取 ⑧account/profile status 枚举加 deleted |
|
|||
|
|
| 2026-05-03 | 浏览器层重构：主实现从 BitBrowserProvider 改为 PlaywrightCliProvider（复用 Neta 已有 playwright-cli skill）；BitBrowser 降为 stub；Entity 字段调整（externalProfileId→sessionName，cdpEndpoint→profileDir/configPath）；接口增加 getCookies/saveState/loadState；不再依赖 playwright Node.js 库 |
|
|||
|
|
| 2026-05-03 | 浏览器层二次重构：澄清"自动化工具"与"浏览器进程"是两个正交层。新增 `BrowserAutomationService`（统一 playwright-cli 包装层）。`IBrowserProvider` 缩减为只管浏览器进程生命周期（去掉 getCookies/saveState/loadState）。S1 主实现重命名为 `PlainChromiumProvider`（playwright-cli + Chromium），BitBrowser/AntBrowser/AdsPower 全部 stub，等具体浏览器需求到位时只增加 provider 不改自动化层 |
|