Files
speaking-practice/docs/superpowers/plans/2026-06-02-speaking-lab-deepseek-subtitle-ai.md
T
2026-06-03 00:34:42 +08:00

22 KiB

Speaking Lab DeepSeek Subtitle AI Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a beginner-friendly DeepSeek-powered admin workflow that cleans English subtitles and generates matching Chinese subtitles.

Architecture: Keep subtitle parsing/cleaning local and deterministic, and use DeepSeek only for translation/text JSON generation. Add a small backend provider boundary so future audio-based ASR timestamp calibration can be added without rewriting the admin UI.

Tech Stack: FastAPI, SQLite, vanilla JavaScript admin page, Python stdlib/httpx if available, Node-based frontend tests.


File Structure

  • Create backend/ai_config.py: environment parsing and AI status helpers.
  • Create backend/subtitle_tools.py: VTT/SRT parsing, duplicate removal, and VTT building.
  • Create backend/deepseek_client.py: DeepSeek chat API client.
  • Create backend/subtitle_ai.py: workflow orchestration.
  • Modify backend/main.py: add AI status/test/process endpoints.
  • Modify backend/db.py: add helper to update/upsert subtitle content for one video.
  • Modify admin.html: add AI subtitle panel.
  • Modify admin.js: call new endpoints and display status/report.
  • Modify styles.css: style the compact admin AI panel.
  • Add Python tests under tests/.
  • Add Node/admin tests if frontend helper logic is extracted.

Task 1: Local Subtitle Tools

Files:

  • Create: backend/subtitle_tools.py

  • Test: tests/subtitle_tools_test.py

  • Step 1: Write the failing test

from backend.subtitle_tools import build_vtt, clean_english_subtitle


def test_clean_english_subtitle_removes_empty_and_duplicate_cues():
    raw = """WEBVTT

00:00:00.000 --> 00:00:00.001
 
00:00:00.872 --> 00:00:04.959
My first job as an investor
was when I was 24 years old,

00:00:00.872 --> 00:00:04.959
My first job as an investor
was when I was 24 years old,
"""

    result = clean_english_subtitle(raw)

    assert result.original_count == 3
    assert result.cleaned_count == 1
    assert result.duplicate_count == 1
    assert result.empty_count == 1
    assert result.cues[0].start == "00:00:00.872"
    assert result.cues[0].end == "00:00:04.959"
    assert result.cues[0].text == "My first job as an investor\nwas when I was 24 years old,"


def test_build_vtt_outputs_numbered_blocks():
    result = build_vtt([
        {
            "start": "00:00:00.872",
            "end": "00:00:04.959",
            "text": "My first job as an investor",
        }
    ])

    assert result == "WEBVTT\n\n1\n00:00:00.872 --> 00:00:04.959\nMy first job as an investor\n"
  • Step 2: Run test to verify it fails

Run: python tests\subtitle_tools_test.py

Expected: FAIL with ModuleNotFoundError or missing function.

  • Step 3: Implement subtitle tools

Create backend/subtitle_tools.py with:

from __future__ import annotations

from dataclasses import dataclass
import re


TIME_RE = re.compile(
    r"(?P<start>\d{1,2}:\d{2}:\d{2}[\.,]\d{3})\s*-->\s*(?P<end>\d{1,2}:\d{2}:\d{2}[\.,]\d{3})"
)


@dataclass
class SubtitleCue:
    start: str
    end: str
    text: str


@dataclass
class CleanSubtitleResult:
    cues: list[SubtitleCue]
    original_count: int
    cleaned_count: int
    duplicate_count: int
    empty_count: int


def normalize_time(value: str) -> str:
    return value.replace(",", ".")


def has_visible_text(rows: list[str]) -> bool:
    text = "".join(rows).replace("&nbsp;", "").replace("\u00a0", "").strip()
    return bool(text)


def parse_subtitle(content: str) -> list[SubtitleCue]:
    cues: list[SubtitleCue] = []
    current_time: tuple[str, str] | None = None
    text_rows: list[str] = []

    def flush() -> None:
        nonlocal current_time, text_rows
        if current_time and text_rows:
            cues.append(SubtitleCue(current_time[0], current_time[1], "\n".join(text_rows).strip()))
        current_time = None
        text_rows = []

    for raw_line in str(content or "").replace("\r", "").replace("\ufeff", "").split("\n"):
        row = raw_line.strip()
        if not row:
            flush()
            continue
        if row.upper().startswith("WEBVTT") or row.isdigit():
            continue
        match = TIME_RE.search(row)
        if match:
            flush()
            current_time = (normalize_time(match.group("start")), normalize_time(match.group("end")))
            continue
        if current_time:
            text_rows.append(row)

    flush()
    return cues


def clean_english_subtitle(content: str) -> CleanSubtitleResult:
    parsed = parse_subtitle(content)
    cleaned: list[SubtitleCue] = []
    seen: set[tuple[str, str, str]] = set()
    duplicate_count = 0
    empty_count = 0

    for cue in parsed:
        rows = cue.text.splitlines()
        if not has_visible_text(rows):
            empty_count += 1
            continue
        key = (cue.start, cue.end, cue.text.strip())
        if key in seen:
            duplicate_count += 1
            continue
        seen.add(key)
        cleaned.append(cue)

    return CleanSubtitleResult(
        cues=cleaned,
        original_count=len(parsed),
        cleaned_count=len(cleaned),
        duplicate_count=duplicate_count,
        empty_count=empty_count,
    )


def build_vtt(cues: list[SubtitleCue] | list[dict[str, str]]) -> str:
    blocks: list[str] = []
    for index, cue in enumerate(cues, 1):
        start = cue.start if isinstance(cue, SubtitleCue) else cue["start"]
        end = cue.end if isinstance(cue, SubtitleCue) else cue["end"]
        text = cue.text if isinstance(cue, SubtitleCue) else cue["text"]
        blocks.append(f"{index}\n{start} --> {end}\n{text.strip()}")
    return "WEBVTT\n\n" + "\n\n".join(blocks).strip() + "\n"
  • Step 4: Run test to verify it passes

Run: python tests\subtitle_tools_test.py

Expected: PASS with exit code 0.

Task 2: AI Config and DeepSeek Client

Files:

  • Create: backend/ai_config.py

  • Create: backend/deepseek_client.py

  • Test: tests/ai_config_test.py

  • Step 1: Write failing config test

import os

from backend.ai_config import get_ai_status, load_deepseek_config


def test_ai_status_reports_missing_key(monkeypatch):
    monkeypatch.delenv("DEEPSEEK_API_KEY", raising=False)

    status = get_ai_status()

    assert status["configured"] is False
    assert "DEEPSEEK_API_KEY" in status["message"]


def test_load_deepseek_config_uses_defaults(monkeypatch):
    monkeypatch.setenv("DEEPSEEK_API_KEY", "test-key")
    monkeypatch.delenv("DEEPSEEK_BASE_URL", raising=False)
    monkeypatch.delenv("DEEPSEEK_MODEL", raising=False)

    config = load_deepseek_config()

    assert config.api_key == "test-key"
    assert config.base_url == "https://api.deepseek.com"
    assert config.model == "deepseek-chat"
  • Step 2: Run test to verify it fails

Run: python tests\ai_config_test.py

Expected: FAIL with missing module.

  • Step 3: Implement config and client

Create backend/ai_config.py:

from __future__ import annotations

from dataclasses import dataclass
import os


@dataclass(frozen=True)
class DeepSeekConfig:
    api_key: str
    base_url: str
    model: str


def load_deepseek_config() -> DeepSeekConfig:
    return DeepSeekConfig(
        api_key=os.getenv("DEEPSEEK_API_KEY", "").strip(),
        base_url=os.getenv("DEEPSEEK_BASE_URL", "https://api.deepseek.com").strip().rstrip("/"),
        model=os.getenv("DEEPSEEK_MODEL", "deepseek-chat").strip(),
    )


def get_ai_status() -> dict[str, object]:
    config = load_deepseek_config()
    if not config.api_key:
        return {
            "configured": False,
            "provider": "deepseek",
            "model": config.model,
            "message": "未配置 DEEPSEEK_API_KEY,请在 .env 或环境变量中填写。",
        }
    return {
        "configured": True,
        "provider": "deepseek",
        "model": config.model,
        "message": "DeepSeek API 已配置。",
    }

Create backend/deepseek_client.py:

from __future__ import annotations

import json
import urllib.error
import urllib.request

from .ai_config import DeepSeekConfig


class DeepSeekError(RuntimeError):
    pass


class DeepSeekClient:
    def __init__(self, config: DeepSeekConfig, timeout_seconds: int = 60):
        self.config = config
        self.timeout_seconds = timeout_seconds

    def chat_json(self, system_prompt: str, user_payload: dict[str, object]) -> dict[str, object]:
        if not self.config.api_key:
            raise DeepSeekError("未配置 DEEPSEEK_API_KEY。")

        body = json.dumps({
            "model": self.config.model,
            "messages": [
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": json.dumps(user_payload, ensure_ascii=False)},
            ],
            "response_format": {"type": "json_object"},
        }).encode("utf-8")
        request = urllib.request.Request(
            f"{self.config.base_url}/chat/completions",
            data=body,
            headers={
                "Authorization": f"Bearer {self.config.api_key}",
                "Content-Type": "application/json",
            },
            method="POST",
        )

        try:
            with urllib.request.urlopen(request, timeout=self.timeout_seconds) as response:
                payload = json.loads(response.read().decode("utf-8"))
        except urllib.error.HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            raise DeepSeekError(f"DeepSeek 请求失败:{error.code} {detail}") from error
        except Exception as error:
            raise DeepSeekError(f"DeepSeek 请求失败:{error}") from error

        content = payload.get("choices", [{}])[0].get("message", {}).get("content", "")
        try:
            return json.loads(content)
        except json.JSONDecodeError as error:
            raise DeepSeekError("DeepSeek 返回内容不是有效 JSON。") from error
  • Step 4: Run config test

Run: python tests\ai_config_test.py

Expected: PASS with exit code 0.

Task 3: Subtitle AI Orchestrator

Files:

  • Create: backend/subtitle_ai.py

  • Test: tests/subtitle_ai_test.py

  • Step 1: Write failing orchestration test

from backend.subtitle_ai import process_subtitles_with_ai


class FakeClient:
    def chat_json(self, system_prompt, user_payload):
        return {
            "translations": [
                {"index": 1, "zh": "我第一次做投资人是在 24 岁。"}
            ]
        }


def test_process_subtitles_with_ai_returns_clean_en_and_zh_vtt():
    raw = """WEBVTT

00:00:00.872 --> 00:00:04.959
My first job as an investor was when I was 24 years old,

00:00:00.872 --> 00:00:04.959
My first job as an investor was when I was 24 years old,
"""

    result = process_subtitles_with_ai(raw, FakeClient())

    assert "My first job as an investor" in result["subtitle_en_content"]
    assert "我第一次做投资人是在 24 岁。" in result["subtitle_zh_content"]
    assert result["report"]["duplicate_count"] == 1
    assert result["report"]["translation_count"] == 1
  • Step 2: Run test to verify it fails

Run: python tests\subtitle_ai_test.py

Expected: FAIL with missing module.

  • Step 3: Implement orchestrator

Create backend/subtitle_ai.py:

from __future__ import annotations

from .subtitle_tools import build_vtt, clean_english_subtitle


TRANSLATION_SYSTEM_PROMPT = """
你是英语学习产品的字幕翻译助手。输入是按顺序排列的英文字幕。
请只返回 JSON,格式为 {"translations":[{"index":1,"zh":"中文翻译"}]}。
要求:中文自然,适合英语学习;不要增删字幕条数;不要解释。
""".strip()


def process_subtitles_with_ai(english_content: str, client) -> dict[str, object]:
    cleaned = clean_english_subtitle(english_content)
    if not cleaned.cues:
        raise ValueError("英文字幕为空,无法处理。")

    payload = {
        "cues": [
            {"index": index, "text": cue.text}
            for index, cue in enumerate(cleaned.cues, 1)
        ]
    }
    response = client.chat_json(TRANSLATION_SYSTEM_PROMPT, payload)
    translations = response.get("translations")
    if not isinstance(translations, list):
        raise ValueError("AI 返回结果缺少 translations。")
    if len(translations) != len(cleaned.cues):
        raise ValueError("AI 返回的翻译条数和英文字幕条数不一致。")

    zh_cues = []
    for cue, translation in zip(cleaned.cues, translations):
        zh_text = str(translation.get("zh", "")).strip()
        if not zh_text:
            raise ValueError("AI 返回了空翻译。")
        zh_cues.append({"start": cue.start, "end": cue.end, "text": zh_text})

    return {
        "subtitle_en_content": build_vtt(cleaned.cues),
        "subtitle_zh_content": build_vtt(zh_cues),
        "report": {
            "original_count": cleaned.original_count,
            "cleaned_count": cleaned.cleaned_count,
            "duplicate_count": cleaned.duplicate_count,
            "empty_count": cleaned.empty_count,
            "translation_count": len(zh_cues),
            "timestamp_calibrated": False,
        },
    }
  • Step 4: Run test

Run: python tests\subtitle_ai_test.py

Expected: PASS with exit code 0.

Task 4: Backend API Endpoints

Files:

  • Modify: backend/db.py

  • Modify: backend/main.py

  • Test: tests/ai_api_test.py

  • Step 1: Add DB helper test

Use an existing SQLite test pattern if present. If no test harness exists, test through the API in Step 2 instead.

  • Step 2: Implement DB helper

In backend/db.py, add:

def upsert_video_subtitle(
    video_id: int,
    *,
    language_code: str,
    language_name: str,
    format: str,
    file_path: str,
    content: str,
) -> None:
    with get_connection() as connection:
        connection.execute(
            """
            INSERT INTO subtitles (
                video_id, language_code, language_name, format, file_path, content
            )
            VALUES (?, ?, ?, ?, ?, ?)
            ON CONFLICT(video_id, language_code) DO UPDATE SET
                language_name = excluded.language_name,
                format = excluded.format,
                file_path = excluded.file_path,
                content = excluded.content,
                updated_at = CURRENT_TIMESTAMP
            """,
            (
                video_id,
                language_code,
                repair_mojibake_text(language_name),
                format,
                file_path,
                repair_mojibake_text(content),
            ),
        )
  • Step 3: Add API endpoints

In backend/main.py, import:

from .ai_config import get_ai_status, load_deepseek_config
from .deepseek_client import DeepSeekClient, DeepSeekError
from .subtitle_ai import process_subtitles_with_ai
from .db import upsert_video_subtitle

Add endpoints before static mounts:

@app.get("/api/ai/status")
def api_ai_status() -> dict[str, object]:
    return get_ai_status()


@app.post("/api/ai/test")
def api_ai_test() -> dict[str, object]:
    status = get_ai_status()
    if not status["configured"]:
        raise HTTPException(status_code=400, detail=status["message"])
    return {"ok": True, "message": "DeepSeek API 配置已读取。"}


@app.post("/api/videos/{video_id}/ai/subtitles")
def api_process_video_subtitles(video_id: int) -> dict[str, object]:
    status = get_ai_status()
    if not status["configured"]:
        raise HTTPException(status_code=400, detail=status["message"])

    video = get_video(video_id)
    if video is None:
        raise HTTPException(status_code=404, detail="视频不存在")
    english = video.get("subtitles", {}).get("en", {}).get("content", "")
    if not english:
        raise HTTPException(status_code=400, detail="请先上传英文字幕。")

    try:
        result = process_subtitles_with_ai(english, DeepSeekClient(load_deepseek_config()))
    except (DeepSeekError, ValueError) as error:
        raise HTTPException(status_code=400, detail=str(error)) from error

    upsert_video_subtitle(
        video_id,
        language_code="en",
        language_name="English",
        format="vtt",
        file_path=video.get("subtitles", {}).get("en", {}).get("filePath", ""),
        content=str(result["subtitle_en_content"]),
    )
    upsert_video_subtitle(
        video_id,
        language_code="zh",
        language_name="中文",
        format="vtt",
        file_path=video.get("subtitles", {}).get("zh", {}).get("filePath", ""),
        content=str(result["subtitle_zh_content"]),
    )

    return {
        "video": get_video(video_id),
        "report": result["report"],
    }
  • Step 4: Run backend syntax checks

Run: python -m py_compile backend\main.py backend\db.py backend\ai_config.py backend\deepseek_client.py backend\subtitle_ai.py backend\subtitle_tools.py

Expected: PASS with exit code 0.

Task 5: Admin UI

Files:

  • Modify: admin.html

  • Modify: admin.js

  • Modify: styles.css

  • Step 1: Add AI panel markup

In admin.html, place near adminSubtitleEditor:

<section class="admin-ai-panel">
  <div>
    <h3>AI 字幕处理</h3>
    <p id="aiSubtitleStatus">正在读取 AI 配置...</p>
  </div>
  <div class="admin-ai-actions">
    <button id="testAiBtn" class="secondary-action" type="button">测试 AI 连接</button>
    <button id="processAiSubtitlesBtn" type="button">AI 修复并翻译字幕</button>
  </div>
  <p id="aiSubtitleReport" class="admin-ai-report" hidden></p>
</section>
  • Step 2: Wire elements in admin.js

Add to els:

aiStatus: IS_BROWSER ? document.querySelector("#aiSubtitleStatus") : null,
aiReport: IS_BROWSER ? document.querySelector("#aiSubtitleReport") : null,
testAiBtn: IS_BROWSER ? document.querySelector("#testAiBtn") : null,
processAiSubtitlesBtn: IS_BROWSER ? document.querySelector("#processAiSubtitlesBtn") : null,

In initializeAdmin() add:

els.testAiBtn?.addEventListener("click", testAiConnection);
els.processAiSubtitlesBtn?.addEventListener("click", processAiSubtitles);
loadAiStatus();

Add functions:

async function loadAiStatus() {
  if (!els.aiStatus) return;
  try {
    const status = await apiRequest("/api/ai/status");
    els.aiStatus.textContent = status.message || "AI 状态未知。";
    els.processAiSubtitlesBtn.disabled = !status.configured || !editingVideo;
  } catch (error) {
    els.aiStatus.textContent = error.message || "AI 状态读取失败。";
    els.processAiSubtitlesBtn.disabled = true;
  }
}

async function testAiConnection() {
  try {
    const result = await apiRequest("/api/ai/test", { method: "POST" });
    showMessage(result.message || "AI 连接正常。");
    await loadAiStatus();
  } catch (error) {
    showMessage(error.message || "AI 连接失败。", true);
  }
}

async function processAiSubtitles() {
  if (!editingVideo) {
    showMessage("请先选择一个素材。", true);
    return;
  }
  els.processAiSubtitlesBtn.disabled = true;
  els.processAiSubtitlesBtn.textContent = "AI 处理中...";
  try {
    const result = await apiRequest(`/api/videos/${editingVideo.id}/ai/subtitles`, { method: "POST" });
    const report = result.report || {};
    els.aiReport.hidden = false;
    els.aiReport.textContent = `处理完成:删除重复 ${report.duplicate_count || 0} 条,删除空白 ${report.empty_count || 0} 条,生成中文 ${report.translation_count || 0} 条。`;
    await loadVideos();
    const updated = videos.find((video) => String(video.id) === String(editingVideo.id));
    if (updated) startEditingVideo(updated);
  } catch (error) {
    showMessage(error.message || "AI 字幕处理失败。", true);
  } finally {
    els.processAiSubtitlesBtn.textContent = "AI 修复并翻译字幕";
    await loadAiStatus();
  }
}

Call loadAiStatus() inside startEditingVideo() and resetFormMode().

  • Step 3: Add CSS

In styles.css, add:

.admin-ai-panel {
  display: grid;
  gap: 10px;
  padding: 12px;
  border: 1px solid var(--line);
  border-radius: 8px;
  background: var(--panel-soft);
}

.admin-ai-panel h3,
.admin-ai-panel p {
  margin: 0;
}

.admin-ai-actions {
  display: flex;
  flex-wrap: wrap;
  gap: 8px;
}

.admin-ai-report {
  color: var(--muted);
  font-size: 13px;
}
  • Step 4: Run frontend checks

Run:

node --check admin.js
node --check app.js

Expected: PASS with exit code 0.

Task 6: Verification

Files:

  • No new files.

  • Step 1: Run Python tests

Run:

python tests\subtitle_tools_test.py
python tests\ai_config_test.py
python tests\subtitle_ai_test.py
python tests\encoding_repair_test.py
python -m py_compile backend\main.py backend\db.py

Expected: all exit 0.

  • Step 2: Run frontend tests

Run:

node --check admin.js
node --check app.js
node --check home.js
node tests\admin-file-input.test.mjs
node tests\admin-subtitle-editor.test.mjs
node tests\progress-helpers.test.mjs

Expected: all exit 0.

  • Step 3: Manual local check without API key

Start the local server and open http://127.0.0.1:8000/admin.html.

Expected:

  • Admin page loads.

  • AI status says 未配置 DEEPSEEK_API_KEY.

  • Clicking AI 修复并翻译字幕 does not crash the page.

  • Step 4: Manual local check with API key

Set environment variable:

$env:DEEPSEEK_API_KEY="your_key_here"

Restart server and open admin page.

Expected:

  • AI status says DeepSeek is configured.
  • Processing a material with English subtitles creates a Chinese subtitle track.
  • Report shows duplicate and translation counts.