Support Bilibili subtitles (including AI-generated Chinese captions) in browser-only mode

Open
#533 5 comments 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

@vincent-peng is already working on this.

Since Oct 24, 2026.

  • #542 by @vincent-peng — open

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
typescript

Research direction

Start by tracing the Chrome extension's browser media acquisition and subtitle extraction paths, then inspect how transcript text and timestamps reach summaries and follow-up chat. Reproduce with a Bilibili video offering AI-generated Chinese captions and check whether the active browser session can retrieve the current part's subtitles. Done means captions are preferred with timestamps, diagnostics distinguish subtitle access from media access, and users can tell when only page text is summarized.

Written by the indexing model from the issue text.

Description

clawsweeper:needs-product-decision clawsweeper:no-new-fix-pr clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster P2

Problem

In the Chrome extension, YouTube summarization works in Direct AI + Browser media mode, but Bilibili video summarization repeatedly fails even when the Bilibili player can display AI-generated Chinese subtitles.

The AI provider connection works through a local OpenAI-compatible endpoint (CLIProxyAPI). The failure appears to occur during content acquisition, before a usable video transcript is available; this is an observation, not a confirmed root cause.

Observed behavior

  1. Open a Bilibili video whose player offers AI-generated Chinese subtitles.
  2. Use the extension with AI connection set to Direct and media runtime set to Browser (no Summarize daemon).
  3. Request a video summary.
  4. The extension reports:

独立模式下无法转录此媒体:The active media source is not fetchable outside the page。请在“运行环境”中选择“守护进程”以支持更多媒体格式。

Browser capture still produces six slide images, but no transcript or OCR text is available. Summarizing page text only does not provide the video's spoken content.

Relevant diagnostic fields from the extension logs:

extractor.try: extractor=url-daemon matched=false
extractor.route.fail: reason=no-extractor-result
slides:payload:applied: slides=6 sourceKind=browser-capture slideRuntime=browser transcriptAvailable=false ocrAvailable=false
slides:text:rebuilt: slides=6 descriptions=6 summaries=0 titles=0 transcriptAvailable=false ocrAvailable=false textMode=transcript source=none

YouTube summaries work in the same setup; a timestamped transcript is available on the tested YouTube page. The exact successful extraction route was not independently verified.

Requested improvement

  • Add or improve Bilibili-specific retrieval of existing subtitles, including AI-generated Chinese captions, preferably within the active browser session and without requiring audio retranscription.
  • Preserve timestamps and select the current video part where applicable.
  • Prefer retrieved subtitle text for summaries and follow-up chat; use audio transcription only as a fallback.
  • Distinguish unavailable/unreadable subtitles from inaccessible media sources in diagnostics, and clarify when the limitation is browser-only support versus a login/permission requirement.
  • Clearly indicate when only title/description/page text is available so it cannot be mistaken for a full video summary.

Scope and missing reproduction details

This request concerns Bilibili browser-mode support, not a claim that all Chinese websites fail. Daemon mode has not been tested. Exact extension/Chrome versions and a public Bilibili reproduction URL have not yet been recorded, so this is submitted as a feature/support request rather than a fully isolated regression report.

If existing Bilibili subtitle support requires a particular setting or permission, documentation of that setup would also help.

Dominant language
TypeScript
Stars
6.9k
Forks
463
Avg merge
20h 29m
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from steipete/summarize

All issues in steipete/summarize

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.