tantingwu/GPU_GUARD_MONOREPO
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
GPU_GUARD_MONOREPO/docs/superpowers/plans/2026-05-04-vehicle-damage-inspection-skill.md

2154 lines
72 KiB
Markdown
Raw Permalink Normal View History

初始化提交
2026-05-20 21:39:12 +08:00
# 汽车环车视频旧伤检测 Skill Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 新增一个生产可用的 `vehicle-damage-inspection` compute-entry Skill，从环车视频中抽帧、检测漆面旧伤、定位标注损伤区域并筛选最佳证据帧。
**Architecture:** 后端先补强 `SkillExecutorService`，为 compute-entry 子进程注入 Windows 安装包生产所需的 `dataDir/workspace/uploads` 非密钥运行时环境变量。业务能力作为一个大 Skill 放在 `packages/backend/skills/vehicle-damage-inspection`，Agent 通过 `execute_skill` 调一次，Skill 内部用 Node CommonJS 脚本按固定流水线执行抽帧、视觉模型检测、bbox 标注和最佳帧筛选。
**Tech Stack:** TypeScript, Midway service, Jest, Node CommonJS, ffmpeg/ffprobe, fluent-ffmpeg, axios, sharp, 火山方舟 OpenAI-compatible Chat Completions, 豆包 `doubao-seed-2-0-pro-260215`
**Spec:** `docs/superpowers/specs/2026-05-04-vehicle-damage-inspection-skill-design.md`
---
## File Map
| 文件 | 职责 | 操作 |
|------|------|------|
| `packages/backend/src/modules/netaclaw/service/skill_runtime_env.ts` | 根据 `pDataPath()` / `pUploadPath()` 构造 compute-entry skill 运行目录 env | 新增 |
| `packages/backend/src/modules/netaclaw/service/skill_env_schema.ts` | 从 `skill.config.yaml env` 同步管理页和 secrets 使用的 env schema | 新增 |
| `packages/backend/src/modules/netaclaw/service/skill_executor.ts` | 注入 runtime env，确保 Windows 安装包产物落在 dataDir | 修改 |
| `packages/backend/test/skill_runtime_env.test.ts` | 测试 runtime env 默认值和 skill secrets 覆盖顺序 | 新增 |
| `packages/backend/test/skill_env_schema.test.ts` | 测试 `skill.config.yaml env` 到 DB `envSchema` 的同步规则 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/SKILL.md` | Agent 可读 Skill 说明 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/skill.config.yaml` | compute-entry 配置、模型 env、依赖、接口 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/README.md` | 人工调试和 Windows 安装包注意事项 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/prompts/damage_detect.md` | 环车旧伤候选检测 prompt | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/prompts/grounding.md` | bbox/grounding prompt | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/prompts/dedup.md` | 损伤去重 prompt | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/prompts/best_frame.md` | 最佳帧筛选 prompt | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/index.cjs` | stdin/stdout JSON 入口和流水线编排 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/make.sh` | `check/run/demo/fix` 本地调试入口 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/setup.ps1` | Windows installer setup 入口；当前 installer 用 `powershell -File` 执行 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/json_utils.cjs` | JSON、bbox、数字 clamp、sleep、并发限制等通用函数 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/workspace.cjs` | task workspace、路径解析、JSON artifact I/O | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/frame_extractor.cjs` | ffmpeg/ffprobe 抽帧 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/vision_client.cjs` | OpenAI-compatible 视觉 API 调用、重试、图片压缩编码 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_detector.cjs` | 分批候选旧伤检测 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/image_marker.cjs` | sharp 画 bbox 标注框 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_grounding.cjs` | 损伤定位标注 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/scripts/lib/best_frame_selector.cjs` | 损伤去重和最佳帧筛选 | 新增 |
| `packages/backend/skills/vehicle-damage-inspection/tests/*.test.cjs` | Skill 纯函数和 mock 流水线测试 | 新增 |
---
### Task 1: Skill env schema 同步
**Files:**
- Create: `packages/backend/src/modules/netaclaw/service/skill_env_schema.ts`
- Modify: `packages/backend/src/modules/netaclaw/service/skill_loader.ts`
- Test: `packages/backend/test/skill_env_schema.test.ts`
- [ ] **Step 1: 写失败测试，覆盖从 skill.config.yaml env 同步默认模型**
Create `packages/backend/test/skill_env_schema.test.ts`:
```typescript
import { buildSkillEnvSchema } from '../src/modules/netaclaw/service/skill_env_schema.js';
describe('buildSkillEnvSchema', () => {
it('uses skill.config.yaml env as the envSchema source of truth', () => {
const schema = buildSkillEnvSchema({
env: [
{
name: 'ARK_API_KEY',
required: true,
description: '火山方舟/豆包 OpenAI-compatible API Key',
},
{
name: 'DAMAGE_DETECT_MODEL',
required: true,
default: 'doubao-seed-2-0-pro-260215',
description: '用于环车旧伤候选检测的多模态模型 ID',
},
],
});
expect(schema).toEqual([
{
name: 'ARK_API_KEY',
required: true,
description: '火山方舟/豆包 OpenAI-compatible API Key',
default: undefined,
},
{
name: 'DAMAGE_DETECT_MODEL',
required: true,
description: '用于环车旧伤候选检测的多模态模型 ID',
default: 'doubao-seed-2-0-pro-260215',
},
]);
});
it('returns null when the skill has no env declaration', () => {
expect(buildSkillEnvSchema({})).toBeNull();
});
});
```
- [ ] **Step 2: 运行测试确认失败**
Run: `pnpm --filter @neta/backend test -- --runInBand test/skill_env_schema.test.ts`
Expected: FAIL with module not found for `skill_env_schema.js`.
- [ ] **Step 3: 新增 skill_env_schema helper**
Create `packages/backend/src/modules/netaclaw/service/skill_env_schema.ts`:
```typescript
import type { SkillConfig } from './skill_config.js';
export interface SkillEnvSchemaItem {
name: string;
required: boolean;
description?: string;
default?: string;
}
export function buildSkillEnvSchema(config: Pick<SkillConfig, 'env'>): SkillEnvSchemaItem[] | null {
if (!Array.isArray(config.env) || config.env.length === 0) return null;
return config.env.map(item => ({
name: item.name,
required: !!item.required,
description: item.description || undefined,
default: item.default || undefined,
}));
}
```
- [ ] **Step 4: 接入 SkillLoaderService.getSkillMetas**
Modify `packages/backend/src/modules/netaclaw/service/skill_loader.ts`.
Add import near existing service imports:
```typescript
import { buildSkillEnvSchema } from './skill_env_schema.js';
```
Inside `getSkillMetas()`, before `await this.skillRepo.save(...)`, add:
```typescript
const config = this.skillConfig.getConfig(fs.name);
const envSchema = buildSkillEnvSchema(config ?? {});
```
Replace the `envSchema` assignment in the save object:
```typescript
envSchema: Array.isArray((fs.metadata as any)?.env) ? (fs.metadata as any).env.map((e: any) => ({
name: e.name,
required: !!e.required,
description: e.description || undefined,
default: e.default || undefined,
})) : null,
```
with:
```typescript
envSchema,
```
This makes `skill.config.yaml env` the single source of truth for model defaults and secret fields. Do not duplicate env declarations in `SKILL.md metadata.env`.
- [ ] **Step 5: 运行测试确认通过**
Run: `pnpm --filter @neta/backend test -- --runInBand test/skill_env_schema.test.ts`
Expected: PASS.
- [ ] **Step 6: 验证后端编译**
Run: `pnpm --filter @neta/backend build`
Expected: build completes without TypeScript errors.
- [ ] **Step 7: 提交**
```bash
git add packages/backend/src/modules/netaclaw/service/skill_env_schema.ts packages/backend/src/modules/netaclaw/service/skill_loader.ts packages/backend/test/skill_env_schema.test.ts
git commit -m "feat(netaclaw): sync skill config env schema"
```
---
### Task 2: 宿主 runtime env 注入
**Files:**
- Create: `packages/backend/src/modules/netaclaw/service/skill_runtime_env.ts`
- Modify: `packages/backend/src/modules/netaclaw/service/skill_executor.ts`
- Test: `packages/backend/test/skill_runtime_env.test.ts`
- [ ] **Step 1: 写失败测试，覆盖 Windows dataDir env 默认值**
Create `packages/backend/test/skill_runtime_env.test.ts`:
```typescript
import * as path from 'path';
import { buildSkillRuntimeEnv } from '../src/modules/netaclaw/service/skill_runtime_env.js';
describe('buildSkillRuntimeEnv', () => {
it('injects dataDir, workspace root and upload root for vehicle damage inspection', () => {
const dataDir = path.join('C:', 'RZYX_AI_DATA');
const uploadRoot = path.join(dataDir, 'uploads');
const env = buildSkillRuntimeEnv({
skillName: 'vehicle-damage-inspection',
dataDir,
uploadRoot,
});
expect(env.RZYX_AI_DATA_DIR).toBe(path.resolve(dataDir));
expect(env.RZYX_AI_UPLOAD_ROOT).toBe(path.resolve(uploadRoot));
expect(env.RZYX_AI_WORKSPACE_ROOT).toBe(
path.join(path.resolve(dataDir), 'workspace', 'vehicle-damage-inspection'),
);
});
it('uses generic workspace directory for other compute-entry skills', () => {
const dataDir = path.join('C:', 'RZYX_AI_DATA');
const env = buildSkillRuntimeEnv({
skillName: 'ocr-reader',
dataDir,
uploadRoot: path.join(dataDir, 'uploads'),
});
expect(env.RZYX_AI_WORKSPACE_ROOT).toBe(
path.join(path.resolve(dataDir), 'workspace', 'ocr-reader'),
);
});
});
```
- [ ] **Step 2: 运行测试确认失败**
Run: `pnpm --filter @neta/backend test -- --runInBand test/skill_runtime_env.test.ts`
Expected: FAIL with module not found for `skill_runtime_env.js`.
- [ ] **Step 3: 新增 runtime env helper**
Create `packages/backend/src/modules/netaclaw/service/skill_runtime_env.ts`:
```typescript
import * as path from 'path';
export interface BuildSkillRuntimeEnvParams {
skillName: string;
dataDir: string;
uploadRoot: string;
}
export function buildSkillRuntimeEnv(params: BuildSkillRuntimeEnvParams): Record<string, string> {
const dataDir = path.resolve(params.dataDir);
const uploadRoot = path.resolve(params.uploadRoot);
const workspaceRoot = path.join(dataDir, 'workspace', params.skillName);
return {
RZYX_AI_DATA_DIR: dataDir,
RZYX_AI_UPLOAD_ROOT: uploadRoot,
RZYX_AI_WORKSPACE_ROOT: workspaceRoot,
};
}
```
- [ ] **Step 4: 接入 SkillExecutorService**
Modify `packages/backend/src/modules/netaclaw/service/skill_executor.ts`.
Add imports near the existing imports:
```typescript
import { pDataPath, pUploadPath } from '../../../comm/path.js';
import { buildSkillRuntimeEnv } from './skill_runtime_env.js';
```
Replace:
```typescript
const skillEnv = await this.skillSecret.resolveEnv(skillName);
const env = { ...baseEnv, ...skillEnv };
```
with:
```typescript
const runtimeEnv = buildSkillRuntimeEnv({
skillName,
dataDir: pDataPath(),
uploadRoot: pUploadPath(),
});
const skillEnv = await this.skillSecret.resolveEnv(skillName);
const env = { ...baseEnv, ...runtimeEnv, ...skillEnv };
```
This order lets skill secrets override runtime defaults when an administrator explicitly configures a path.
- [ ] **Step 5: 运行测试确认通过**
Run: `pnpm --filter @neta/backend test -- --runInBand test/skill_runtime_env.test.ts`
Expected: PASS.
- [ ] **Step 6: 验证后端编译**
Run: `pnpm --filter @neta/backend build`
Expected: build completes without TypeScript errors.
- [ ] **Step 7: 提交**
```bash
git add packages/backend/src/modules/netaclaw/service/skill_runtime_env.ts packages/backend/src/modules/netaclaw/service/skill_executor.ts packages/backend/test/skill_runtime_env.test.ts
git commit -m "feat(netaclaw): inject runtime data paths into compute skills"
```
---
### Task 3: Skill scaffold and metadata
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/SKILL.md`
- Create: `packages/backend/skills/vehicle-damage-inspection/skill.config.yaml`
- Create: `packages/backend/skills/vehicle-damage-inspection/README.md`
- Create: `packages/backend/skills/vehicle-damage-inspection/prompts/damage_detect.md`
- Create: `packages/backend/skills/vehicle-damage-inspection/prompts/grounding.md`
- Create: `packages/backend/skills/vehicle-damage-inspection/prompts/dedup.md`
- Create: `packages/backend/skills/vehicle-damage-inspection/prompts/best_frame.md`
- [ ] **Step 1: 创建 SKILL.md**
Create `packages/backend/skills/vehicle-damage-inspection/SKILL.md`:
```markdown
---
name: vehicle-damage-inspection
description: >
汽车环车视频旧伤检测。输入一个环车视频文件或 /upload/... 路径，自动抽帧、
检测车身漆面旧伤、定位 bbox 并输出红框标注图和最佳证据帧。
license: MIT
metadata:
version: "1.0.0"
category: vehicle-inspection
skillType: multimodal
tags: ["vehicle", "video", "damage", "bbox"]
---
# 汽车环车视频旧伤检测
当用户需要从汽车环车视频中识别漆面旧伤、划痕、凹陷、掉漆、锈蚀等外观损伤时，使用此 skill。
这是 compute-entry skill。不要手动拆步骤调用；直接使用 `execute_skill`：
```json
{
"name": "vehicle-damage-inspection",
"input": {
"videoUrl": "/upload/20260504/car.mp4",
"fps": 3,
"topN": 1
}
}
```
## 输入
- `videoUrl`: 必填。视频本地路径或 `/upload/...` 路径；第一版不支持 http(s) URL。
- `taskId`: 可选。未传时自动生成。
- `fps`: 可选，默认 3。
- `quality`: 可选，默认 90。
- `batchSize`: 可选，默认 12。
- `concurrency`: 可选，默认 2。
- `groundingWindow`: 可选，默认 1.5 秒。
- `topN`: 可选，默认每处损伤 1 张最佳帧。
- `mode`: 可选，`full`、`frames-only` 或 `detect-only`。
## 输出
返回 JSON，包含 `success`、`taskId`、`workspacePath`、`summary`、`damages` 和 `artifacts`。
生产环境中，所有检测产物写入后端 dataDir 下的 `workspace/vehicle-damage-inspection/{taskId}`。
```
- [ ] **Step 2: 创建 skill.config.yaml**
Create `packages/backend/skills/vehicle-damage-inspection/skill.config.yaml`:
```yaml
runtime: node
entrypoint: scripts/index.cjs
timeout: 900000
dependencies:
system:
- name: ffmpeg
check: "ffmpeg -version"
- name: ffprobe
check: "ffprobe -version"
- name: node
check: "node --version"
node:
packages:
- axios
- sharp
- fluent-ffmpeg
- "@ffmpeg-installer/ffmpeg"
- "@ffprobe-installer/ffprobe"
setup:
posix: scripts/make.sh fix
win32: scripts/setup.ps1
env:
- name: ARK_API_KEY
required: true
description: 火山方舟/豆包 OpenAI-compatible API Key
- name: ARK_API_URL
required: false
default: https://ark.cn-beijing.volces.com/api/v3/chat/completions
description: OpenAI-compatible chat completions endpoint
- name: DAMAGE_DETECT_MODEL
required: true
default: doubao-seed-2-0-pro-260215
description: 用于环车旧伤候选检测的多模态模型 ID
- name: DAMAGE_GROUNDING_MODEL
required: true
default: doubao-seed-2-0-pro-260215
description: 支持 bbox/grounding 输出的视觉模型 ID
- name: BEST_FRAME_MODEL
required: false
default: doubao-seed-2-0-pro-260215
description: 用于损伤去重和最佳帧筛选的模型 ID
- name: RZYX_AI_WORKSPACE_ROOT
required: false
description: 生产环境由宿主注入，默认 dataDir/workspace/vehicle-damage-inspection
- name: RZYX_AI_UPLOAD_ROOT
required: false
description: 生产环境由宿主注入，默认 dataDir/uploads
- name: RZYX_AI_DATA_DIR
required: false
description: 后端运行时 dataDir，由宿主注入
interface:
input:
videoUrl:
type: string
required: true
description: 视频本地路径或 /upload/... 路径；第一版不支持 http(s) URL
taskId:
type: string
required: false
description: 可选任务 ID
fps:
type: number
required: false
default: "3"
description: 抽帧帧率
quality:
type: number
required: false
default: "90"
description: 输出 JPG 质量，范围 1-100
batchSize:
type: number
required: false
default: "12"
description: 每批发送给检测模型的帧数
concurrency:
type: number
required: false
default: "2"
description: 多模态 API 并发数
groundingWindow:
type: number
required: false
default: "1.5"
description: 每个损伤时间点前后取帧窗口，单位秒
topN:
type: number
required: false
default: "1"
description: 每处损伤返回的最佳证据帧数量
mode:
type: string
required: false
default: full
description: full | frames-only | detect-only
output:
success:
type: boolean
taskId:
type: string
workspacePath:
type: string
summary:
type: object
damages:
type: array
artifacts:
type: object
```
- [ ] **Step 3: 创建 prompt 文件**
Create `packages/backend/skills/vehicle-damage-inspection/prompts/damage_detect.md`:
```markdown
你是专业汽车环车视频旧伤检测专家。请检查给定视频帧中的车身外观可见区域，识别漆面旧伤，包括划痕、凹陷、掉漆、裂纹、锈蚀。
边界：
- 只把可见且明确的车身外观损伤列为 damages。
- 反光、污渍、阴影、雨水、压缩噪声、拍摄模糊不要直接判定为损伤。
- 玻璃、轮胎、车灯、车牌等非漆面问题如明显可见，可标记为 non_paint_damage，但不要混入 paint damage。
- 不确定时设置 uncertain=true，confidence="low"。
只输出 JSON：
{
"damages": [
{
"time_second": 12.4,
"part": "左前门",
"type": "划痕",
"severity": "轻微",
"description": "左前门中部可见细长白色划痕",
"confidence": "medium",
"uncertain": false
}
],
"summary": "本批检测简述"
}
```
Create `packages/backend/skills/vehicle-damage-inspection/prompts/grounding.md`:
```markdown
请在图中定位指定汽车外观损伤区域，输出 bbox。bbox 坐标使用 0-1000 归一化坐标，格式：
[12.40 second] <bbox>x1 y1 x2 y2</bbox>
如果某帧看不到该损伤，不要输出该帧 bbox。
```
Create `packages/backend/skills/vehicle-damage-inspection/prompts/dedup.md`:
```markdown
你是汽车旧伤记录去重专家。以下损伤记录可能来自连续视频帧中的同一处物理损伤。
合并规则：
- 位置相同或相近、类型相同，且时间相近，视为同一处物理损伤。
- 位置描述略有差异但明显指同一部位，也应合并。
- 不同部位或不同损伤类型不要合并。
只输出 JSON：
{
"groups": [
{
"merged_part": "左前门",
"merged_type": "划痕",
"merged_severity": "轻微",
"merged_description": "左前门中部细长划痕",
"member_indices": [0, 2]
}
]
}
```
Create `packages/backend/skills/vehicle-damage-inspection/prompts/best_frame.md`:
```markdown
以下是同一处汽车外观损伤的多张标注图。请选出最适合作为证据展示的帧。
优先级：
1. 红框准确框住损伤，不偏移、不过大、不遗漏。
2. 损伤区域清晰，不模糊、不遮挡。
3. 光照和角度能看清损伤全貌。
4. 避免反光、过曝、过暗。
只输出 JSON：
{
"best_timestamps": [12.4],
"reasons": ["红框准确，划痕清晰可见"]
}
```
- [ ] **Step 4: 创建 README.md**
Create `packages/backend/skills/vehicle-damage-inspection/README.md`:
```markdown
# vehicle-damage-inspection
生产形态：Windows 安装包中由后端复制到 `{dataDir}/skills/vehicle-damage-inspection` 后执行。
运行产物必须写入 `{dataDir}/workspace/vehicle-damage-inspection/{taskId}`，不要写入 skill 代码目录。
## 本地检查
```bash
bash scripts/make.sh check
```
## 本地运行
```bash
ARK_API_KEY=xxx bash scripts/make.sh run --video /path/to/car.mp4 --fps 3
```
## 必需模型
默认模型为 `doubao-seed-2-0-pro-260215`，用于检测、bbox 定位和最佳帧筛选。
```
- [ ] **Step 5: 验证 Skill Loader 能识别配置**
Run: `pnpm --filter @neta/backend test -- --runInBand test/entity_exports.test.ts`
Expected: PASS. This is a smoke check that backend tests still load.
- [ ] **Step 6: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection
git commit -m "feat(skills): scaffold vehicle damage inspection skill"
```
---
### Task 4: JSON utilities and workspace module
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/json_utils.cjs`
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/workspace.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/json_utils.test.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/workspace.test.cjs`
- [ ] **Step 1: 写 json_utils 测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/json_utils.test.cjs`:
```javascript
const assert = require('node:assert');
const { parseJsonLoose, parseBboxes, clampNumber, runWithConcurrency } = require('../scripts/lib/json_utils.cjs');
(async () => {
assert.deepStrictEqual(parseJsonLoose('```json\n{"a":1}\n```'), { a: 1 });
assert.deepStrictEqual(parseBboxes('[1.0 second] <bbox>10 20 30 40</bbox>'), [
{ x1: 10, y1: 20, x2: 30, y2: 40 },
]);
assert.strictEqual(clampNumber('9', 1, 5, 3), 5);
assert.strictEqual(clampNumber(undefined, 1, 5, 3), 3);
let active = 0;
let maxActive = 0;
const tasks = Array.from({ length: 5 }, (_, i) => async () => {
active += 1;
maxActive = Math.max(maxActive, active);
await new Promise(resolve => setTimeout(resolve, 5));
active -= 1;
return i;
});
const result = await runWithConcurrency(tasks, 2);
assert.deepStrictEqual(result, [0, 1, 2, 3, 4]);
assert.ok(maxActive <= 2);
})();
```
- [ ] **Step 2: 写 workspace 测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/workspace.test.cjs`:
```javascript
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { createWorkspace, writeJson, readJson, resolveVideoPath } = require('../scripts/lib/workspace.cjs');
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'vehicle-skill-workspace-'));
const uploads = path.join(root, 'uploads');
fs.mkdirSync(path.join(uploads, '20260504'), { recursive: true });
const video = path.join(uploads, '20260504', 'car.mp4');
fs.writeFileSync(video, 'fake');
try {
const ws = createWorkspace({ taskId: 'task-a', workspaceRoot: path.join(root, 'workspace') });
assert.strictEqual(ws.taskId, 'task-a');
assert.ok(fs.existsSync(path.join(ws.workspacePath, 'frames')));
writeJson(ws, 'video_info.json', { ok: true });
assert.deepStrictEqual(readJson(ws, 'video_info.json'), { ok: true });
assert.strictEqual(resolveVideoPath('/upload/20260504/car.mp4', { uploadRoot: uploads }), video);
assert.throws(() => resolveVideoPath('/upload/../secret.mp4', { uploadRoot: uploads }), /Invalid upload path/);
assert.throws(() => resolveVideoPath('https://example.test/car.mp4', { uploadRoot: uploads }), /Remote video URLs are not supported/);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
```
- [ ] **Step 3: 运行测试确认失败**
Run:
```bash
node packages/backend/skills/vehicle-damage-inspection/tests/json_utils.test.cjs
node packages/backend/skills/vehicle-damage-inspection/tests/workspace.test.cjs
```
Expected: both FAIL with missing module errors.
- [ ] **Step 4: 实现 json_utils.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/json_utils.cjs`:
```javascript
function parseJsonLoose(text) {
if (typeof text !== 'string') return null;
let value = text.trim();
if (value.startsWith('```')) {
value = value.split('\n').filter(line => !line.trim().startsWith('```')).join('\n').trim();
}
try {
return JSON.parse(value);
} catch {
const start = value.indexOf('{');
const end = value.lastIndexOf('}');
if (start >= 0 && end > start) {
try {
return JSON.parse(value.slice(start, end + 1));
} catch {
return null;
}
}
return null;
}
}
function parseBboxes(text) {
const result = [];
const pattern = /<bbox>\s*(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s*<\/bbox>/g;
let match;
while ((match = pattern.exec(String(text || ''))) !== null) {
result.push({
x1: Number(match[1]),
y1: Number(match[2]),
x2: Number(match[3]),
y2: Number(match[4]),
});
}
return result;
}
function clampNumber(value, min, max, fallback) {
const n = Number(value);
if (!Number.isFinite(n)) return fallback;
return Math.max(min, Math.min(max, n));
}
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async function runWithConcurrency(tasks, limit) {
const results = new Array(tasks.length);
let nextIndex = 0;
async function worker() {
while (nextIndex < tasks.length) {
const index = nextIndex++;
results[index] = await tasks[index]();
}
}
await Promise.all(Array.from({ length: Math.min(Math.max(1, limit), tasks.length) }, () => worker()));
return results;
}
module.exports = {
parseJsonLoose,
parseBboxes,
clampNumber,
sleep,
runWithConcurrency,
};
```
- [ ] **Step 5: 实现 workspace.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/workspace.cjs`:
```javascript
const fs = require('node:fs');
const path = require('node:path');
const crypto = require('node:crypto');
function sanitizeTaskId(taskId) {
const id = taskId || `${new Date().toISOString().replace(/[-:.TZ]/g, '').slice(0, 14)}-${crypto.randomUUID().slice(0, 8)}`;
if (!/^[a-zA-Z0-9_-]+$/.test(id)) throw new Error(`Invalid taskId: ${id}`);
return id;
}
function createWorkspace({ taskId, workspaceRoot }) {
const root = path.resolve(workspaceRoot);
const id = sanitizeTaskId(taskId);
const workspacePath = path.join(root, id);
fs.mkdirSync(path.join(workspacePath, 'source'), { recursive: true });
fs.mkdirSync(path.join(workspacePath, 'frames'), { recursive: true });
fs.mkdirSync(path.join(workspacePath, 'marked_frames'), { recursive: true });
return { taskId: id, root, workspacePath };
}
function ensureInside(root, candidate) {
const rootPath = path.resolve(root);
const fullPath = path.resolve(candidate);
if (fullPath !== rootPath && !fullPath.startsWith(rootPath + path.sep)) {
throw new Error(`Path escapes root: ${candidate}`);
}
return fullPath;
}
function writeJson(workspace, relativePath, value) {
const filePath = ensureInside(workspace.workspacePath, path.join(workspace.workspacePath, relativePath));
fs.mkdirSync(path.dirname(filePath), { recursive: true });
fs.writeFileSync(filePath, JSON.stringify(value, null, 2), 'utf8');
return filePath;
}
function readJson(workspace, relativePath) {
const filePath = ensureInside(workspace.workspacePath, path.join(workspace.workspacePath, relativePath));
if (!fs.existsSync(filePath)) return null;
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
}
function resolveVideoPath(videoUrl, options) {
if (!videoUrl || typeof videoUrl !== 'string') throw new Error('videoUrl is required');
if (fs.existsSync(videoUrl)) return path.resolve(videoUrl);
const uploadMatch = videoUrl.match(/^\/upload\/(.+)$/);
if (uploadMatch) {
if (!options.uploadRoot) throw new Error('RZYX_AI_UPLOAD_ROOT is required for /upload paths');
const relative = uploadMatch[1].replace(/[\\/]+/g, path.sep);
if (relative.split(path.sep).includes('..')) throw new Error('Invalid upload path');
const filePath = ensureInside(options.uploadRoot, path.join(options.uploadRoot, relative));
if (!fs.existsSync(filePath)) throw new Error(`Video file not found: ${filePath}`);
return filePath;
}
if (/^https?:\/\//i.test(videoUrl)) {
throw new Error('Remote video URLs are not supported in the first version; use a local path or /upload/... path');
}
throw new Error(`Video file not found: ${videoUrl}`);
}
module.exports = {
createWorkspace,
writeJson,
readJson,
resolveVideoPath,
ensureInside,
};
```
- [ ] **Step 6: 运行测试确认通过**
Run:
```bash
node packages/backend/skills/vehicle-damage-inspection/tests/json_utils.test.cjs
node packages/backend/skills/vehicle-damage-inspection/tests/workspace.test.cjs
```
Expected: both commands exit 0.
- [ ] **Step 7: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/json_utils.cjs packages/backend/skills/vehicle-damage-inspection/scripts/lib/workspace.cjs packages/backend/skills/vehicle-damage-inspection/tests/json_utils.test.cjs packages/backend/skills/vehicle-damage-inspection/tests/workspace.test.cjs
git commit -m "feat(skills): add vehicle inspection workspace utilities"
```
---
### Task 5: Frame extraction module
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/frame_extractor.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/frame_extractor.test.cjs`
- [ ] **Step 1: 写 ffmpeg command builder 测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/frame_extractor.test.cjs`:
```javascript
const assert = require('node:assert');
const { buildFrameName, qualityToFfmpegQv, estimateFrameTimestamp } = require('../scripts/lib/frame_extractor.cjs');
assert.strictEqual(buildFrameName(1, 0), 'frame_000001_0.00s.jpg');
assert.strictEqual(buildFrameName(38, 12.4), 'frame_000038_12.40s.jpg');
assert.strictEqual(qualityToFfmpegQv(100), 1);
assert.strictEqual(qualityToFfmpegQv(1), 31);
assert.strictEqual(estimateFrameTimestamp(6, 3), 2);
```
- [ ] **Step 2: 运行测试确认失败**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/frame_extractor.test.cjs`
Expected: FAIL with missing module.
- [ ] **Step 3: 实现 frame_extractor.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/frame_extractor.cjs`:
```javascript
const fs = require('node:fs');
const path = require('node:path');
const ffmpeg = require('fluent-ffmpeg');
const ffmpegInstaller = require('@ffmpeg-installer/ffmpeg');
const ffprobeInstaller = require('@ffprobe-installer/ffprobe');
const { writeJson } = require('./workspace.cjs');
ffmpeg.setFfmpegPath(process.env.FFMPEG_PATH || ffmpegInstaller.path);
ffmpeg.setFfprobePath(process.env.FFPROBE_PATH || ffprobeInstaller.path);
function buildFrameName(index, timestamp) {
return `frame_${String(index).padStart(6, '0')}_${Number(timestamp).toFixed(2)}s.jpg`;
}
function qualityToFfmpegQv(quality) {
const q = Math.max(1, Math.min(100, Number(quality) || 90));
return Math.max(1, Math.min(31, Math.round(31 - (q / 100) * 30)));
}
function estimateFrameTimestamp(indexZeroBased, fps) {
return Number((indexZeroBased / fps).toFixed(2));
}
function probeVideo(videoPath) {
return new Promise((resolve, reject) => {
ffmpeg.ffprobe(videoPath, (err, data) => err ? reject(err) : resolve(data));
});
}
async function extractFrames({ workspace, videoPath, fps, quality }) {
const framesDir = path.join(workspace.workspacePath, 'frames');
fs.rmSync(framesDir, { recursive: true, force: true });
fs.mkdirSync(framesDir, { recursive: true });
const probe = await probeVideo(videoPath);
const stream = (probe.streams || []).find(item => item.codec_type === 'video');
if (!stream) throw new Error('No video stream found');
const duration = Number(probe.format && probe.format.duration ? probe.format.duration : 0);
const width = stream.width || 0;
const height = stream.height || 0;
const fpsValue = Number(fps) || 3;
const qv = qualityToFfmpegQv(quality);
const tempPattern = path.join(framesDir, 'raw_%06d.jpg');
await new Promise((resolve, reject) => {
ffmpeg(videoPath)
.outputOptions(['-vf', `fps=${fpsValue}`, '-q:v', String(qv)])
.output(tempPattern)
.on('end', resolve)
.on('error', reject)
.run();
});
const rawFiles = fs.readdirSync(framesDir).filter(name => /^raw_\d+\.jpg$/.test(name)).sort();
if (rawFiles.length === 0) throw new Error('No frames extracted');
const frames = rawFiles.map((name, index) => {
const timestamp = estimateFrameTimestamp(index, fpsValue);
const finalName = buildFrameName(index + 1, timestamp);
fs.renameSync(path.join(framesDir, name), path.join(framesDir, finalName));
return {
index,
timestamp,
relativePath: `frames/${finalName}`,
path: path.join(framesDir, finalName),
};
});
const videoInfo = {
duration: Number(duration.toFixed(2)),
videoFps: stream.r_frame_rate || '',
resolution: `${width}x${height}`,
extractFps: fpsValue,
extractedFrames: frames.length,
};
writeJson(workspace, 'video_info.json', {
videoPath,
videoInfo,
frames: frames.map(({ index, timestamp, relativePath }) => ({ index, timestamp, relativePath })),
});
return { videoInfo, frames };
}
module.exports = {
buildFrameName,
qualityToFfmpegQv,
estimateFrameTimestamp,
extractFrames,
};
```
- [ ] **Step 4: 运行测试确认通过**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/frame_extractor.test.cjs`
Expected: exits 0.
- [ ] **Step 5: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/frame_extractor.cjs packages/backend/skills/vehicle-damage-inspection/tests/frame_extractor.test.cjs
git commit -m "feat(skills): add vehicle video frame extractor"
```
---
### Task 6: Vision client module
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/vision_client.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/vision_client.test.cjs`
- [ ] **Step 1: 写 vision client 纯函数测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/vision_client.test.cjs`:
```javascript
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { resolveModelConfig, encodeImageDataUri, createTimestampedImageContent } = require('../scripts/lib/vision_client.cjs');
const env = {
ARK_API_KEY: 'key',
ARK_API_URL: 'https://example.test/v1/chat/completions',
DAMAGE_DETECT_MODEL: 'detect-model',
DAMAGE_GROUNDING_MODEL: 'ground-model',
};
assert.deepStrictEqual(resolveModelConfig(env), {
apiKey: 'key',
apiUrl: 'https://example.test/v1/chat/completions',
detectModel: 'detect-model',
groundingModel: 'ground-model',
bestFrameModel: 'detect-model',
});
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vision-client-'));
try {
const img = path.join(dir, 'a.jpg');
fs.writeFileSync(img, Buffer.from([1, 2, 3]));
assert.strictEqual(encodeImageDataUri(img), 'data:image/jpeg;base64,AQID');
const content = createTimestampedImageContent([{ timestamp: 1.25, path: img }]);
assert.strictEqual(content[0].text, '[1.25 second]');
assert.ok(content[1].image_url.url.startsWith('data:image/jpeg;base64,'));
} finally {
fs.rmSync(dir, { recursive: true, force: true });
}
```
- [ ] **Step 2: 运行测试确认失败**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/vision_client.test.cjs`
Expected: FAIL with missing module.
- [ ] **Step 3: 实现 vision_client.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/vision_client.cjs`:
```javascript
const fs = require('node:fs');
const axios = require('axios');
const { sleep } = require('./json_utils.cjs');
function resolveModelConfig(env) {
const apiKey = env.ARK_API_KEY;
if (!apiKey) throw new Error('Missing required env: ARK_API_KEY');
const apiUrl = env.ARK_API_URL || 'https://ark.cn-beijing.volces.com/api/v3/chat/completions';
const detectModel = env.DAMAGE_DETECT_MODEL || 'doubao-seed-2-0-pro-260215';
const groundingModel = env.DAMAGE_GROUNDING_MODEL || detectModel;
const bestFrameModel = env.BEST_FRAME_MODEL || detectModel;
return { apiKey, apiUrl, detectModel, groundingModel, bestFrameModel };
}
function encodeImageDataUri(filePath) {
const buffer = fs.readFileSync(filePath);
return `data:image/jpeg;base64,${buffer.toString('base64')}`;
}
function createTimestampedImageContent(frames) {
const content = [];
for (const frame of frames) {
content.push({ type: 'text', text: `[${Number(frame.timestamp).toFixed(2)} second]` });
content.push({ type: 'image_url', image_url: { url: encodeImageDataUri(frame.path) } });
}
return content;
}
async function callVisionApi({ config, model, messages, maxTokens = 4096, temperature = 0.1, timeoutMs = 600000, maxRetries = 3 }) {
let lastError = '';
for (let attempt = 0; attempt <= maxRetries; attempt += 1) {
try {
const response = await axios.post(
config.apiUrl,
{ model, messages, max_tokens: maxTokens, temperature },
{
headers: {
Authorization: `Bearer ${config.apiKey}`,
'Content-Type': 'application/json',
},
timeout: timeoutMs,
maxBodyLength: Infinity,
maxContentLength: Infinity,
},
);
return {
ok: true,
content: response.data && response.data.choices && response.data.choices[0] && response.data.choices[0].message
? response.data.choices[0].message.content || ''
: '',
usage: response.data ? response.data.usage : undefined,
};
} catch (error) {
const status = error.response && error.response.status;
const body = error.response && error.response.data ? JSON.stringify(error.response.data).slice(0, 500) : error.message;
lastError = `API调用失败(${status || 'network'}): ${body}`;
const retryable = status === 429 || status === 500 || status === 502 || status === 503 || status === 504;
if (retryable && attempt < maxRetries) {
await sleep(1000 * Math.pow(2, attempt));
continue;
}
break;
}
}
return { ok: false, error: lastError };
}
module.exports = {
resolveModelConfig,
encodeImageDataUri,
createTimestampedImageContent,
callVisionApi,
};
```
- [ ] **Step 4: 运行测试确认通过**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/vision_client.test.cjs`
Expected: exits 0.
- [ ] **Step 5: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/vision_client.cjs packages/backend/skills/vehicle-damage-inspection/tests/vision_client.test.cjs
git commit -m "feat(skills): add doubao vision client for vehicle inspection"
```
---
### Task 7: Damage candidate detector
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_detector.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/damage_detector.test.cjs`
- [ ] **Step 1: 写 mock 检测测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/damage_detector.test.cjs`:
```javascript
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { createWorkspace, readJson } = require('../scripts/lib/workspace.cjs');
const { detectDamageCandidates } = require('../scripts/lib/damage_detector.cjs');
(async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'damage-detector-'));
try {
const workspace = createWorkspace({ taskId: 'task-detect', workspaceRoot: root });
const framePath = path.join(workspace.workspacePath, 'frames', 'frame_000001_0.00s.jpg');
fs.writeFileSync(framePath, Buffer.from([1, 2, 3]));
const frames = [{ index: 0, timestamp: 0, path: framePath, relativePath: 'frames/frame_000001_0.00s.jpg' }];
const mockVision = async () => ({
ok: true,
content: JSON.stringify({
damages: [{ time_second: 0, part: '左前门', type: '划痕', severity: '轻微', description: '细长划痕', confidence: 'medium' }],
summary: '发现一处损伤',
}),
});
const result = await detectDamageCandidates({
workspace,
frames,
modelConfig: { detectModel: 'm', apiKey: 'k', apiUrl: 'u' },
prompt: 'prompt',
batchSize: 1,
concurrency: 1,
callVisionApi: mockVision,
});
assert.strictEqual(result.candidates.length, 1);
assert.strictEqual(result.candidates[0].id, 'cand_001');
assert.strictEqual(readJson(workspace, 'damage_candidates.json').candidates.length, 1);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
})();
```
- [ ] **Step 2: 运行测试确认失败**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/damage_detector.test.cjs`
Expected: FAIL with missing module.
- [ ] **Step 3: 实现 damage_detector.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_detector.cjs`:
```javascript
const { parseJsonLoose, runWithConcurrency } = require('./json_utils.cjs');
const { createTimestampedImageContent, callVisionApi: defaultCallVisionApi } = require('./vision_client.cjs');
const { writeJson } = require('./workspace.cjs');
function normalizeCandidate(raw, index, batchNumber) {
return {
id: `cand_${String(index + 1).padStart(3, '0')}`,
timestamp: Number(raw.time_second ?? raw.timestamp ?? 0),
part: raw.part || raw.location || '',
type: raw.type || '',
severity: raw.severity || '',
description: raw.description || '',
confidence: raw.confidence || 'medium',
uncertain: Boolean(raw.uncertain),
batch: batchNumber,
};
}
async function detectDamageCandidates(params) {
const {
workspace,
frames,
modelConfig,
prompt,
batchSize,
concurrency,
callVisionApi = defaultCallVisionApi,
} = params;
const batches = [];
for (let start = 0; start < frames.length; start += batchSize) {
batches.push(frames.slice(start, start + batchSize));
}
const batchRecords = [];
const collected = [];
await runWithConcurrency(batches.map((batch, batchIndex) => async () => {
const content = [
{ type: 'text', text: prompt },
...createTimestampedImageContent(batch),
{ type: 'text', text: '请只输出 JSON。' },
];
const result = await callVisionApi({
config: modelConfig,
model: modelConfig.detectModel,
messages: [{ role: 'user', content }],
});
const record = {
batch: batchIndex + 1,
frameStart: batch[0].index,
frameEnd: batch[batch.length - 1].index,
status: result.ok ? 'success' : 'error',
error: result.ok ? undefined : result.error,
};
batchRecords[batchIndex] = record;
if (!result.ok) return;
const parsed = parseJsonLoose(result.content || '');
const damages = Array.isArray(parsed && parsed.damages) ? parsed.damages : [];
for (const damage of damages) {
collected.push({ damage, batchNumber: batchIndex + 1 });
}
}), concurrency);
const candidates = collected.map((item, index) => normalizeCandidate(item.damage, index, item.batchNumber));
const output = {
totalFrames: frames.length,
batches: batchRecords,
candidates,
};
writeJson(workspace, 'damage_candidates.json', output);
return output;
}
module.exports = {
detectDamageCandidates,
normalizeCandidate,
};
```
- [ ] **Step 4: 运行测试确认通过**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/damage_detector.test.cjs`
Expected: exits 0.
- [ ] **Step 5: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_detector.cjs packages/backend/skills/vehicle-damage-inspection/tests/damage_detector.test.cjs
git commit -m "feat(skills): detect vehicle damage candidates from frames"
```
---
### Task 8: Image marker and damage grounding
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/image_marker.cjs`
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_grounding.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/image_marker.test.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/damage_grounding.test.cjs`
- [ ] **Step 1: 写 image marker 测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/image_marker.test.cjs`:
```javascript
const assert = require('node:assert');
const { normalizedToPixel } = require('../scripts/lib/image_marker.cjs');
assert.deepStrictEqual(
normalizedToPixel({ x1: 100, y1: 200, x2: 300, y2: 400 }, { width: 1000, height: 500 }),
{ x1: 100, y1: 100, x2: 300, y2: 200 },
);
```
- [ ] **Step 2: 写 grounding mock 测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/damage_grounding.test.cjs`:
```javascript
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { createWorkspace, readJson } = require('../scripts/lib/workspace.cjs');
const { groundDamages } = require('../scripts/lib/damage_grounding.cjs');
(async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'damage-grounding-'));
try {
const workspace = createWorkspace({ taskId: 'task-ground', workspaceRoot: root });
const framePath = path.join(workspace.workspacePath, 'frames', 'frame_000001_0.00s.jpg');
fs.writeFileSync(framePath, Buffer.from([1, 2, 3]));
const frames = [{ index: 0, timestamp: 0, path: framePath, relativePath: 'frames/frame_000001_0.00s.jpg' }];
const candidates = [{ id: 'cand_001', timestamp: 0, part: '左前门', type: '划痕', severity: '轻微', description: '细长划痕' }];
const mockVision = async () => ({ ok: true, content: '[0.00 second] <bbox>10 20 30 40</bbox>' });
const mockMarker = async ({ dstPath }) => {
fs.writeFileSync(dstPath, 'marked');
return true;
};
const result = await groundDamages({
workspace,
frames,
candidates,
modelConfig: { groundingModel: 'm', apiKey: 'k', apiUrl: 'u' },
prompt: 'prompt',
groundingWindow: 1.5,
callVisionApi: mockVision,
drawBbox: mockMarker,
});
assert.strictEqual(result.annotations.length, 1);
assert.strictEqual(result.annotations[0].markedFrames.length, 1);
assert.strictEqual(readJson(workspace, 'damage_annotations.json').annotations.length, 1);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
})();
```
- [ ] **Step 3: 运行测试确认失败**
Run:
```bash
node packages/backend/skills/vehicle-damage-inspection/tests/image_marker.test.cjs
node packages/backend/skills/vehicle-damage-inspection/tests/damage_grounding.test.cjs
```
Expected: both FAIL with missing module.
- [ ] **Step 4: 实现 image_marker.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/image_marker.cjs`:
```javascript
const fs = require('node:fs');
const path = require('node:path');
const sharp = require('sharp');
function normalizedToPixel(box, metadata) {
const width = metadata.width || 1;
const height = metadata.height || 1;
return {
x1: Math.round((box.x1 * width) / 1000),
y1: Math.round((box.y1 * height) / 1000),
x2: Math.round((box.x2 * width) / 1000),
y2: Math.round((box.y2 * height) / 1000),
};
}
async function drawBbox({ srcPath, dstPath, bboxes, label }) {
fs.mkdirSync(path.dirname(dstPath), { recursive: true });
const metadata = await sharp(srcPath).metadata();
const width = metadata.width || 1;
const height = metadata.height || 1;
const parts = [];
for (const box of bboxes) {
const pixel = normalizedToPixel(box, { width, height });
const rectWidth = Math.max(1, pixel.x2 - pixel.x1);
const rectHeight = Math.max(1, pixel.y2 - pixel.y1);
parts.push(`<rect x="${pixel.x1}" y="${pixel.y1}" width="${rectWidth}" height="${rectHeight}" fill="none" stroke="red" stroke-width="4"/>`);
if (label) {
const safeLabel = String(label).replace(/[<>&"]/g, '');
parts.push(`<text x="${pixel.x1}" y="${Math.max(18, pixel.y1 - 6)}" font-size="18" fill="red">${safeLabel}</text>`);
}
}
const overlay = Buffer.from(`<svg width="${width}" height="${height}" xmlns="http://www.w3.org/2000/svg">${parts.join('')}</svg>`);
await sharp(srcPath).composite([{ input: overlay, top: 0, left: 0 }]).jpeg({ quality: 90 }).toFile(dstPath);
return true;
}
module.exports = {
normalizedToPixel,
drawBbox,
};
```
- [ ] **Step 5: 实现 damage_grounding.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_grounding.cjs`:
```javascript
const fs = require('node:fs');
const path = require('node:path');
const { parseBboxes } = require('./json_utils.cjs');
const { createTimestampedImageContent, callVisionApi: defaultCallVisionApi } = require('./vision_client.cjs');
const { drawBbox: defaultDrawBbox, normalizedToPixel } = require('./image_marker.cjs');
const { writeJson } = require('./workspace.cjs');
function selectNearbyFrames(frames, timestamp, windowSeconds) {
return frames.filter(frame => Math.abs(Number(frame.timestamp) - Number(timestamp)) <= windowSeconds);
}
async function groundDamages(params) {
const {
workspace,
frames,
candidates,
modelConfig,
prompt,
groundingWindow,
callVisionApi = defaultCallVisionApi,
drawBbox = defaultDrawBbox,
} = params;
const annotations = [];
for (let i = 0; i < candidates.length; i += 1) {
const candidate = candidates[i];
const nearbyFrames = selectNearbyFrames(frames, candidate.timestamp, groundingWindow);
const content = [
...createTimestampedImageContent(nearbyFrames),
{ type: 'text', text: `${prompt}\n损伤描述：${candidate.part} ${candidate.type} ${candidate.description}` },
];
const result = await callVisionApi({
config: modelConfig,
model: modelConfig.groundingModel,
messages: [{ role: 'user', content }],
maxTokens: 2048,
temperature: 0.1,
});
const bboxes = result.ok ? parseBboxes(result.content || '') : [];
const markedFrames = [];
for (const frame of nearbyFrames) {
const markedName = `dmg_${String(i + 1).padStart(3, '0')}_${Number(frame.timestamp).toFixed(2)}s.jpg`;
const dstPath = path.join(workspace.workspacePath, 'marked_frames', markedName);
if (bboxes.length > 0) {
await drawBbox({ srcPath: frame.path, dstPath, bboxes, label: `${candidate.part}-${candidate.type}` });
} else {
fs.copyFileSync(frame.path, dstPath);
}
const pixel = bboxes[0] ? normalizedToPixel(bboxes[0], { width: 1000, height: 1000 }) : null;
markedFrames.push({
timestamp: frame.timestamp,
sourceRelativePath: frame.relativePath,
markedRelativePath: `marked_frames/${markedName}`,
path: dstPath,
bbox: bboxes[0] ? { normalized: bboxes[0], pixel } : null,
groundingStatus: bboxes.length > 0 ? 'bbox' : 'no_bbox',
groundingRaw: result.ok ? result.content : result.error,
});
}
annotations.push({
candidateId: candidate.id,
damageId: `dmg_${String(i + 1).padStart(3, '0')}`,
part: candidate.part,
type: candidate.type,
severity: candidate.severity,
description: candidate.description,
markedFrames,
});
}
const output = { annotations };
writeJson(workspace, 'damage_annotations.json', output);
return output;
}
module.exports = {
selectNearbyFrames,
groundDamages,
};
```
- [ ] **Step 6: 运行测试确认通过**
Run:
```bash
node packages/backend/skills/vehicle-damage-inspection/tests/image_marker.test.cjs
node packages/backend/skills/vehicle-damage-inspection/tests/damage_grounding.test.cjs
```
Expected: both commands exit 0.
- [ ] **Step 7: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/image_marker.cjs packages/backend/skills/vehicle-damage-inspection/scripts/lib/damage_grounding.cjs packages/backend/skills/vehicle-damage-inspection/tests/image_marker.test.cjs packages/backend/skills/vehicle-damage-inspection/tests/damage_grounding.test.cjs
git commit -m "feat(skills): ground and mark vehicle damage frames"
```
---
### Task 9: Best frame selector
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/lib/best_frame_selector.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/best_frame_selector.test.cjs`
- [ ] **Step 1: 写最佳帧筛选测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/best_frame_selector.test.cjs`:
```javascript
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { createWorkspace, readJson } = require('../scripts/lib/workspace.cjs');
const { heuristicFrameScore, selectBestFrames, buildBestFramePrompt } = require('../scripts/lib/best_frame_selector.cjs');
(async () => {
assert.ok(heuristicFrameScore({ bbox: { normalized: { x1: 100, y1: 100, x2: 300, y2: 300 } } }) > heuristicFrameScore({ bbox: null }));
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'best-frame-'));
try {
const workspace = createWorkspace({ taskId: 'task-best', workspaceRoot: root });
const annotations = [{
damageId: 'dmg_001',
part: '左前门',
type: '划痕',
severity: '轻微',
description: '细长划痕',
markedFrames: [
{ timestamp: 0, markedRelativePath: 'marked_frames/a.jpg', path: 'a.jpg', bbox: null },
{ timestamp: 1, markedRelativePath: 'marked_frames/b.jpg', path: 'b.jpg', bbox: { normalized: { x1: 100, y1: 100, x2: 300, y2: 300 } } },
],
}];
const result = await selectBestFrames({
workspace,
annotations,
topN: 1,
modelConfig: { bestFrameModel: 'doubao-seed-2-0-pro-260215' },
prompt: '选择最佳证据帧',
callVisionApi: async () => ({ ok: true, content: '{"best_timestamps":[1],"reasons":["bbox准确"]}' }),
});
assert.ok(buildBestFramePrompt(annotations[0], '选择最佳证据帧').includes('dmg_001'));
assert.strictEqual(result.damages[0].bestFrames[0].timestamp, 1);
assert.strictEqual(result.damages[0].bestFrames[0].selectionReason, 'bbox准确');
assert.strictEqual(readJson(workspace, 'best_frames.json').bestFrameCount, 1);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
})();
```
- [ ] **Step 2: 运行测试确认失败**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/best_frame_selector.test.cjs`
Expected: FAIL with missing module.
- [ ] **Step 3: 实现 best_frame_selector.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/lib/best_frame_selector.cjs`:
```javascript
const { writeJson } = require('./workspace.cjs');
const { parseJsonLoose } = require('./json_utils.cjs');
const { callVisionApi: defaultCallVisionApi } = require('./vision_client.cjs');
function heuristicFrameScore(frame) {
let score = 0;
if (frame.bbox && frame.bbox.normalized) {
score += 100;
const box = frame.bbox.normalized;
const area = Math.max(0, box.x2 - box.x1) * Math.max(0, box.y2 - box.y1);
if (area >= 10000 && area <= 250000) score += 30;
if (box.x1 > 10 && box.y1 > 10 && box.x2 < 990 && box.y2 < 990) score += 20;
}
return score;
}
function dedupeByTimestamp(frames) {
const map = new Map();
for (const frame of frames) {
const key = Number(frame.timestamp).toFixed(2);
const existing = map.get(key);
if (!existing || heuristicFrameScore(frame) > heuristicFrameScore(existing)) {
map.set(key, frame);
}
}
return Array.from(map.values());
}
function buildBestFramePrompt(annotation, prompt) {
const frames = (annotation.markedFrames || []).map(frame => ({
timestamp: frame.timestamp,
hasBbox: !!frame.bbox,
bbox: frame.bbox ? frame.bbox.normalized : null,
}));
return `${prompt}\n损伤ID：${annotation.damageId}\n部位：${annotation.part}\n类型：${annotation.type}\n描述：${annotation.description}\n候选帧：${JSON.stringify(frames)}`;
}
function parseBestFrameSelection(content) {
const parsed = parseJsonLoose(content || '');
if (!parsed || !Array.isArray(parsed.best_timestamps)) return { timestamps: [], reasons: [] };
return {
timestamps: parsed.best_timestamps.map(Number).filter(Number.isFinite),
reasons: Array.isArray(parsed.reasons) ? parsed.reasons.map(String) : [],
};
}
async function chooseFramesWithModel({ annotation, candidates, topN, modelConfig, prompt, callVisionApi }) {
if (!modelConfig || !modelConfig.bestFrameModel || candidates.length <= 1) return null;
const result = await callVisionApi({
config: modelConfig,
model: modelConfig.bestFrameModel,
messages: [{ role: 'user', content: [{ type: 'text', text: buildBestFramePrompt(annotation, prompt) }] }],
maxTokens: 1024,
temperature: 0.1,
});
if (!result.ok) return null;
const selection = parseBestFrameSelection(result.content);
const picked = [];
for (const timestamp of selection.timestamps) {
const frame = candidates.find(item => Math.abs(Number(item.timestamp) - timestamp) < 0.01);
if (frame && !picked.includes(frame)) picked.push(frame);
if (picked.length >= topN) break;
}
if (picked.length === 0) return null;
return picked.map((frame, index) => ({ frame, reason: selection.reasons[index] || 'BEST_FRAME_MODEL selected this frame' }));
}
async function selectBestFrames({ workspace, annotations, topN, modelConfig, prompt = '', callVisionApi = defaultCallVisionApi }) {
const damages = [];
for (let index = 0; index < annotations.length; index += 1) {
const annotation = annotations[index];
const candidates = dedupeByTimestamp(annotation.markedFrames || []);
const modelSelected = await chooseFramesWithModel({ annotation, candidates, topN, modelConfig, prompt, callVisionApi });
const selected = modelSelected || candidates
.slice()
.sort((a, b) => heuristicFrameScore(b) - heuristicFrameScore(a))
.slice(0, topN)
.map(frame => ({ frame, reason: 'heuristic fallback' }));
const bestFrames = selected.map(({ frame, reason }) => ({
timestamp: frame.timestamp,
path: frame.path,
relativePath: frame.markedRelativePath,
bbox: frame.bbox ? frame.bbox.normalized : null,
selectionReason: reason,
}));
damages.push({
id: annotation.damageId || `dmg_${String(index + 1).padStart(3, '0')}`,
part: annotation.part,
type: annotation.type,
severity: annotation.severity,
description: annotation.description,
timestamps: candidates.map(frame => frame.timestamp),
bestFrames,
});
}
const bestFrameCount = damages.reduce((sum, item) => sum + item.bestFrames.length, 0);
const output = {
damageCount: damages.length,
bestFrameCount,
damages,
};
writeJson(workspace, 'best_frames.json', output);
return output;
}
module.exports = {
heuristicFrameScore,
dedupeByTimestamp,
buildBestFramePrompt,
parseBestFrameSelection,
selectBestFrames,
};
```
- [ ] **Step 4: 运行测试确认通过**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/best_frame_selector.test.cjs`
Expected: exits 0.
- [ ] **Step 5: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/lib/best_frame_selector.cjs packages/backend/skills/vehicle-damage-inspection/tests/best_frame_selector.test.cjs
git commit -m "feat(skills): select best vehicle damage evidence frames"
```
---
### Task 10: Compute-entry pipeline entrypoint
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/index.cjs`
- Test: `packages/backend/skills/vehicle-damage-inspection/tests/index_frames_only.test.cjs`
- [ ] **Step 1: 写 frames-only 入口测试**
Create `packages/backend/skills/vehicle-damage-inspection/tests/index_frames_only.test.cjs`:
```javascript
const assert = require('node:assert');
const { normalizeInput, buildFinalOutput } = require('../scripts/index.cjs');
const input = normalizeInput({ videoUrl: '/upload/a.mp4', fps: '20', quality: '120', mode: 'frames-only' });
assert.strictEqual(input.videoUrl, '/upload/a.mp4');
assert.strictEqual(input.fps, 10);
assert.strictEqual(input.quality, 100);
assert.strictEqual(input.mode, 'frames-only');
const output = buildFinalOutput({
workspace: { taskId: 'task', workspacePath: 'C:/data/workspace/task' },
videoInfo: { duration: 1, resolution: '1x1', extractedFrames: 3 },
candidates: [],
damages: [],
artifacts: { videoInfo: 'video_info.json' },
});
assert.strictEqual(output.success, true);
assert.strictEqual(output.summary.frameCount, 3);
```
- [ ] **Step 2: 运行测试确认失败**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/index_frames_only.test.cjs`
Expected: FAIL with missing module.
- [ ] **Step 3: 实现 scripts/index.cjs**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/index.cjs`:
```javascript
const fs = require('node:fs');
const path = require('node:path');
const { clampNumber } = require('./lib/json_utils.cjs');
const { createWorkspace, resolveVideoPath, writeJson } = require('./lib/workspace.cjs');
const { extractFrames } = require('./lib/frame_extractor.cjs');
const { resolveModelConfig } = require('./lib/vision_client.cjs');
const { detectDamageCandidates } = require('./lib/damage_detector.cjs');
const { groundDamages } = require('./lib/damage_grounding.cjs');
const { selectBestFrames } = require('./lib/best_frame_selector.cjs');
function normalizeInput(raw) {
if (!raw || typeof raw !== 'object') throw new Error('Input JSON object is required');
if (!raw.videoUrl) throw new Error('videoUrl is required');
return {
videoUrl: String(raw.videoUrl),
taskId: raw.taskId ? String(raw.taskId) : undefined,
fps: clampNumber(raw.fps, 0.2, 10, 3),
quality: clampNumber(raw.quality, 1, 100, 90),
batchSize: clampNumber(raw.batchSize, 1, 20, 12),
concurrency: clampNumber(raw.concurrency, 1, 4, 2),
groundingWindow: clampNumber(raw.groundingWindow, 0.5, 5, 1.5),
topN: clampNumber(raw.topN, 1, 5, 1),
mode: ['full', 'frames-only', 'detect-only'].includes(raw.mode) ? raw.mode : 'full',
};
}
function readPrompt(name) {
return fs.readFileSync(path.join(__dirname, '..', 'prompts', name), 'utf8');
}
function resolveWorkspaceRoot(env) {
if (env.RZYX_AI_WORKSPACE_ROOT) return env.RZYX_AI_WORKSPACE_ROOT;
if (env.RZYX_AI_DATA_DIR) return path.join(env.RZYX_AI_DATA_DIR, 'workspace', 'vehicle-damage-inspection');
if (env.NODE_ENV === 'production') throw new Error('RZYX_AI_DATA_DIR or RZYX_AI_WORKSPACE_ROOT is required in production');
return path.join(__dirname, '..', '.workspace');
}
function buildFinalOutput({ workspace, videoInfo, candidates, damages, artifacts }) {
const bestFrameCount = damages.reduce((sum, item) => sum + (item.bestFrames || []).length, 0);
return {
success: true,
taskId: workspace.taskId,
workspacePath: workspace.workspacePath,
summary: {
duration: videoInfo.duration,
resolution: videoInfo.resolution,
frameCount: videoInfo.extractedFrames,
candidateDamageCount: candidates.length,
mergedDamageCount: damages.length,
bestFrameCount,
},
damages,
artifacts,
};
}
async function run(rawInput, env = process.env) {
const input = normalizeInput(rawInput);
const workspaceRoot = resolveWorkspaceRoot(env);
const workspace = createWorkspace({ taskId: input.taskId, workspaceRoot });
const uploadRoot = env.RZYX_AI_UPLOAD_ROOT || (env.RZYX_AI_DATA_DIR ? path.join(env.RZYX_AI_DATA_DIR, 'uploads') : undefined);
const videoPath = resolveVideoPath(input.videoUrl, { uploadRoot });
const { videoInfo, frames } = await extractFrames({
workspace,
videoPath,
fps: input.fps,
quality: input.quality,
});
if (input.mode === 'frames-only') {
const output = buildFinalOutput({
workspace,
videoInfo,
candidates: [],
damages: [],
artifacts: { videoInfo: 'video_info.json' },
});
writeJson(workspace, 'run_summary.json', output);
return output;
}
const modelConfig = resolveModelConfig(env);
const detection = await detectDamageCandidates({
workspace,
frames,
modelConfig,
prompt: readPrompt('damage_detect.md'),
batchSize: input.batchSize,
concurrency: input.concurrency,
});
if (input.mode === 'detect-only' || detection.candidates.length === 0) {
const output = buildFinalOutput({
workspace,
videoInfo,
candidates: detection.candidates,
damages: [],
artifacts: { videoInfo: 'video_info.json', damageCandidates: 'damage_candidates.json' },
});
writeJson(workspace, 'run_summary.json', output);
return output;
}
const grounding = await groundDamages({
workspace,
frames,
candidates: detection.candidates,
modelConfig,
prompt: readPrompt('grounding.md'),
groundingWindow: input.groundingWindow,
});
const best = await selectBestFrames({
workspace,
annotations: grounding.annotations,
topN: input.topN,
modelConfig,
prompt: readPrompt('best_frame.md'),
});
const output = buildFinalOutput({
workspace,
videoInfo,
candidates: detection.candidates,
damages: best.damages,
artifacts: {
videoInfo: 'video_info.json',
damageCandidates: 'damage_candidates.json',
damageAnnotations: 'damage_annotations.json',
bestFrames: 'best_frames.json',
runSummary: 'run_summary.json',
},
});
writeJson(workspace, 'run_summary.json', output);
return output;
}
async function main() {
try {
const stdin = fs.readFileSync(0, 'utf8');
const input = JSON.parse(stdin || '{}');
const output = await run(input);
process.stdout.write(JSON.stringify(output));
} catch (error) {
process.stdout.write(JSON.stringify({ success: false, error: error.message }));
process.exitCode = 0;
}
}
if (require.main === module) {
main();
}
module.exports = {
normalizeInput,
buildFinalOutput,
run,
};
```
- [ ] **Step 4: 运行测试确认通过**
Run: `node packages/backend/skills/vehicle-damage-inspection/tests/index_frames_only.test.cjs`
Expected: exits 0.
- [ ] **Step 5: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/index.cjs packages/backend/skills/vehicle-damage-inspection/tests/index_frames_only.test.cjs
git commit -m "feat(skills): wire vehicle inspection compute entrypoint"
```
---
### Task 11: make.sh and Windows setup diagnostics
**Files:**
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/make.sh`
- Create: `packages/backend/skills/vehicle-damage-inspection/scripts/setup.ps1`
- [ ] **Step 1: 创建 make.sh**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/make.sh`:
```bash
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cmd_check() {
local ok=true
command -v node >/dev/null 2>&1 || { echo "node missing"; ok=false; }
command -v ffmpeg >/dev/null 2>&1 || echo "ffmpeg not found in PATH; npm installer fallback may be used"
command -v ffprobe >/dev/null 2>&1 || echo "ffprobe not found in PATH; npm installer fallback may be used"
node -e "require('axios'); require('sharp'); require('fluent-ffmpeg')" || ok=false
$ok
}
cmd_fix() {
cd "$ROOT"
npm install axios sharp fluent-ffmpeg @ffmpeg-installer/ffmpeg @ffprobe-installer/ffprobe
}
cmd_run() {
local video=""
local fps="3"
local mode="full"
while [[ $# -gt 0 ]]; do
case "$1" in
--video) video="$2"; shift 2 ;;
--fps) fps="$2"; shift 2 ;;
--mode) mode="$2"; shift 2 ;;
*) echo "Unknown option: $1"; exit 1 ;;
esac
done
if [[ -z "$video" ]]; then
echo "Usage: bash scripts/make.sh run --video FILE [--fps 3] [--mode full]"
exit 1
fi
printf '{"videoUrl":%s,"fps":%s,"mode":%s}\n' \
"$(node -e "process.stdout.write(JSON.stringify(process.argv[1]))" "$video")" \
"$fps" \
"$(node -e "process.stdout.write(JSON.stringify(process.argv[1]))" "$mode")" \
| node "$ROOT/scripts/index.cjs"
}
cmd_tests() {
for file in "$ROOT"/tests/*.test.cjs; do
node "$file"
done
}
case "${1:-}" in
check) cmd_check ;;
fix) cmd_fix ;;
run) shift; cmd_run "$@" ;;
test) cmd_tests ;;
*) echo "Usage: bash scripts/make.sh check|fix|run|test"; exit 1 ;;
esac
```
- [ ] **Step 2: 运行本地测试集合**
Run: `bash packages/backend/skills/vehicle-damage-inspection/scripts/make.sh test`
Expected: every `*.test.cjs` exits 0.
- [ ] **Step 3: 运行 check**
Run: `bash packages/backend/skills/vehicle-damage-inspection/scripts/make.sh check`
Expected: exits 0 if node packages are installed. If it fails because packages are not installed in the skill directory, run `bash packages/backend/skills/vehicle-damage-inspection/scripts/make.sh fix`, then rerun check.
- [ ] **Step 4: 提交**
- [ ] **Step 4: 创建 Windows setup.ps1**
Create `packages/backend/skills/vehicle-damage-inspection/scripts/setup.ps1`:
```powershell
$ErrorActionPreference = "Stop"
$SkillRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
Set-Location $SkillRoot
npm install axios sharp fluent-ffmpeg @ffmpeg-installer/ffmpeg @ffprobe-installer/ffprobe
```
- [ ] **Step 5: 验证 Windows setup 脚本语法**
Run: `powershell -NoProfile -ExecutionPolicy Bypass -File packages/backend/skills/vehicle-damage-inspection/scripts/setup.ps1`
Expected: npm dependencies install successfully in the skill directory. For production packaging, run this during build/install, not during every skill execution.
- [ ] **Step 6: 提交**
```bash
git add packages/backend/skills/vehicle-damage-inspection/scripts/make.sh packages/backend/skills/vehicle-damage-inspection/scripts/setup.ps1
git commit -m "feat(skills): add vehicle inspection skill cli"
```
---
### Task 12: End-to-end backend integration validation
**Files:**
- Test only: no source files expected unless validation exposes a bug.
- [ ] **Step 1: 运行 Skill 纯测试**
Run: `bash packages/backend/skills/vehicle-damage-inspection/scripts/make.sh test`
Expected: exits 0.
- [ ] **Step 2: 运行后端 runtime env 测试**
Run: `pnpm --filter @neta/backend test -- --runInBand test/skill_runtime_env.test.ts`
Expected: PASS.
- [ ] **Step 3: 运行后端 build**
Run: `pnpm --filter @neta/backend build`
Expected: build completes without TypeScript errors.
- [ ] **Step 4: 模拟 Windows dataDir 路径运行 frames-only**
Prepare a small MP4 file. If no sample is available, use ffmpeg to generate one:
```bash
mkdir -p C:/tmp/rzyx-ai-skill-test/uploads/20260504
ffmpeg -f lavfi -i testsrc=duration=2:size=320x240:rate=10 C:/tmp/rzyx-ai-skill-test/uploads/20260504/sample.mp4 -y
```
Run:
```bash
RZYX_AI_DATA_DIR=C:/tmp/rzyx-ai-skill-test \
RZYX_AI_UPLOAD_ROOT=C:/tmp/rzyx-ai-skill-test/uploads \
RZYX_AI_WORKSPACE_ROOT=C:/tmp/rzyx-ai-skill-test/workspace/vehicle-damage-inspection \
bash packages/backend/skills/vehicle-damage-inspection/scripts/make.sh run \
--video /upload/20260504/sample.mp4 \
--fps 1 \
--mode frames-only
```
Expected:
- stdout is valid JSON with `"success":true`.
- `C:/tmp/rzyx-ai-skill-test/workspace/vehicle-damage-inspection/.../video_info.json` exists.
- No `workspace` directory is created under `packages/backend/skills/vehicle-damage-inspection` during this production-path simulation.
- [ ] **Step 5: 提交 validation fixes only if needed**
If any validation step required code fixes, first inspect the exact modified files:
```bash
git status --short
```
Then add only files related to this skill implementation. The expected fix scope is one or more of these paths:
```bash
git add packages/backend/src/modules/netaclaw/service/skill_runtime_env.ts packages/backend/src/modules/netaclaw/service/skill_executor.ts packages/backend/test/skill_runtime_env.test.ts packages/backend/skills/vehicle-damage-inspection
git commit -m "fix(skills): stabilize vehicle inspection integration"
```
If no files changed, skip this commit.
---
## Self-Review Checklist
- [x] Spec coverage: Tasks cover host env injection, Skill metadata, prompts, workspace protocol, frame extraction, vision client, detection, grounding, image marking, best-frame selection, compute entrypoint, CLI, tests, and Windows dataDir validation.
- [x] Placeholder scan: The plan contains no unresolved markers, no undefined file paths, and no deferred-implementation instructions.
- [x] Type consistency: The public Skill name is consistently `vehicle-damage-inspection`; runtime env names are consistently `RZYX_AI_DATA_DIR`, `RZYX_AI_WORKSPACE_ROOT`, and `RZYX_AI_UPLOAD_ROOT`; default model is consistently `doubao-seed-2-0-pro-260215`.
Reference in New Issue Copy Permalink