[video] Direct vendor APIs — Vidu and Alibaba DashScope (Wan)

オープン
#5,176 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
14/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
静か
技術スタック
python
領域
ai, api

調査の方向性

The work belongs in praisonaiagents/video/adapters/, where new files vidu.py and dashscope_wan.py are proposed alongside a shared _async_task_poll.py; start by reading the existing video adapters there and how the registry resolves model prefixes. Done means one Vidu route and two DashScope Wan routes (wan2.6-flash and wan2.2-kf2v-flash, which must reject seconds other than 5) pass mocked tests, with DASHSCOPE_API_KEY and VIDU_API_KEY added to .env.example. The issue spans several providers and open design decisions about off-peak job handling, so it needs maintainer scoping first.

索引モデルが issue の本文から書いたものです。

説明

bug claude documentation

Epic: #5171


Problem statement

The cost picker uses direct vendor APIs to avoid reseller markup:

Vidu Q3 Turbo
  • Channel: Vidu direct (off-peak)
  • 1080p, includes audio
  • Off-peak 48 hour queue vs standard queue ~$0.065/s (picker)
Alibaba Wan (DashScope)
SKU Picker behavior
Wan 2.6 Flash Direct pricing vs OpenRouter ~$0.15/s resold
Wan 2.2 KF2V Flash 5s fixed, silent, ~$0.015/s @ 480p
Wan 2.7 Strong water/leaves motion; loose loop seam on people

PraisonAI TS maps qwen → DASHSCOPE_API_KEY for chat embeddings, not Wan video REST endpoints. No ViduVideoProvider exists.


Vidu API — implementation requirements

Provider id: vidu

Model strings:

VideoAgent(llm="vidu/q3-turbo")
VideoAgent(llm="vidu/q3-turbo-offpeak")  # if separate SKU

Tasks:

  1. Read Vidu official API docs at implementation time (endpoints for text-to-video / image-to-video).

  2. Auth header pattern (likely Authorization: Bearer <API_KEY> — env VIDU_API_KEY).

  3. Queue mode parameter for off-peak vs standard — map kwargs:

    agent.generate(..., queue="offpeak")  # or vendor enum
    
  4. Poll until terminal state; download MP4.

  5. User-facing SLA text: off-peak may take up to 48 hours (picker); do not block wait_for_completion default 600s — require max_wait_time override or async job pattern:

    Design decision: For off-peak, generate() returns job id immediately; document that user must poll with long max_wait_time or use webhooks if Vidu supports them.

Error codes: map 402/429/quota to clear messages.


Alibaba DashScope Wan — implementation requirements

Provider id: dashscope (preferred prefix for consistency with Qwen)

Model strings (illustrative — pin exact API model ids in PR):

VideoAgent(llm="dashscope/wan2.6-flash")
VideoAgent(llm="dashscope/wan2.2-kf2v-flash")
VideoAgent(llm="dashscope/wan2.7")

Environment:

  • DASHSCOPE_API_KEY (same as Qwen)

API pattern (typical for DashScope async tasks):

  1. POST create async task with model, input, parameters
  2. Poll GET task status
  3. Retrieve output.video_url

Implement against current Alibaba documentation for Wan video (not text generation).

KF2V Flash rules:

  • If user passes seconds != "5", raise VideoProviderValidationError or clamp with warning (picker: fixed 5s).
  • Silent output — no audio track in deliverable.

Resolution mapping:

Picker Parameter
480p resolution="480P" or equivalent
720p / 1080p map from size or kwargs

Channel selection (document in matrix #10)

For Wan 2.7 same picker row may list OpenRouter + Alibaba direct:

llm= prefix Channel
dashscope/wan2.7 Direct (lower $/s per picker)
openrouter/... Single key multi-model (#03)
replicate/... Flat-run variants (#04)

Registry must treat these as different providers with different job id namespaces.


ViduVideoProvider / DashScopeWanVideoProvider files

praisonaiagents/video/adapters/vidu.py
praisonaiagents/video/adapters/dashscope_wan.py

Shared utilities:

  • praisonaiagents/video/adapters/_async_task_poll.py — generic poll loop with backoff if both vendors share pattern

Testing

Vidu mocks:

  • off-peak submit returns status=queued
  • standard queue faster completion

DashScope mocks:

  • KF2V rejects wrong duration
  • 2.6 Flash success path

No live keys in CI.


Examples

  • examples/python/video/video_vidu_q3_turbo_offpeak.py
  • examples/python/video/video_dashscope_wan_kf2v.py
  • examples/python/video/video_dashscope_wan27.py

Acceptance criteria

  • One Vidu + two Alibaba routes working in mocked tests
  • Off-peak documented with long-running job guidance
  • Env vars in .env.example
  • Estimator direct $/s from catalog (#08) differs from OpenRouter same SKU

Dependencies

Requires #02

Related

Epic #01, #03, #04, #08, #10


Appendix: Vidu off-peak job UX

  • Return job id immediately; document expected queue latency up to 48h.
  • Provide estimate_queue_wait() in catalog, not in provider.
  • Desktop (#09) should show 'queued' state without blocking UI thread for 48h.

Appendix: DashScope Wan SKU checklist

  1. wan2.6-flash — map seconds and resolution from VideoAgent args.
  2. wan2.2-kf2v-flash — enforce 5s duration.
  3. wan2.7 — document loop seam caveat from picker measurements.
  4. wan2.6-flash — map seconds and resolution from VideoAgent args.
  5. wan2.2-kf2v-flash — enforce 5s duration.
  6. wan2.7 — document loop seam caveat from picker measurements.
  7. wan2.6-flash — map seconds and resolution from VideoAgent args.
  8. wan2.2-kf2v-flash — enforce 5s duration.
  9. wan2.7 — document loop seam caveat from picker measurements.
  10. wan2.6-flash — map seconds and resolution from VideoAgent args.
  11. wan2.2-kf2v-flash — enforce 5s duration.
  12. wan2.7 — document loop seam caveat from picker measurements.
  13. wan2.6-flash — map seconds and resolution from VideoAgent args.
  14. wan2.2-kf2v-flash — enforce 5s duration.
  15. wan2.7 — document loop seam caveat from picker measurements.
  16. wan2.6-flash — map seconds and resolution from VideoAgent args.
  17. wan2.2-kf2v-flash — enforce 5s duration.
  18. wan2.7 — document loop seam caveat from picker measurements.
  19. wan2.6-flash — map seconds and resolution from VideoAgent args.
  20. wan2.2-kf2v-flash — enforce 5s duration.
  21. wan2.7 — document loop seam caveat from picker measurements.
  22. wan2.6-flash — map seconds and resolution from VideoAgent args.
  23. wan2.2-kf2v-flash — enforce 5s duration.
  24. wan2.7 — document loop seam caveat from picker measurements.
  25. wan2.6-flash — map seconds and resolution from VideoAgent args.
  26. wan2.2-kf2v-flash — enforce 5s duration.
  27. wan2.7 — document loop seam caveat from picker measurements.
  28. wan2.6-flash — map seconds and resolution from VideoAgent args.
  29. wan2.2-kf2v-flash — enforce 5s duration.
  30. wan2.7 — document loop seam caveat from picker measurements.
  31. wan2.6-flash — map seconds and resolution from VideoAgent args.
  32. wan2.2-kf2v-flash — enforce 5s duration.
  33. wan2.7 — document loop seam caveat from picker measurements.
  34. wan2.6-flash — map seconds and resolution from VideoAgent args.
  35. wan2.2-kf2v-flash — enforce 5s duration.
  36. wan2.7 — document loop seam caveat from picker measurements.
  37. wan2.6-flash — map seconds and resolution from VideoAgent args.
  38. wan2.2-kf2v-flash — enforce 5s duration.
  39. wan2.7 — document loop seam caveat from picker measurements.
  40. wan2.6-flash — map seconds and resolution from VideoAgent args.
  41. wan2.2-kf2v-flash — enforce 5s duration.
  42. wan2.7 — document loop seam caveat from picker measurements.
  43. wan2.6-flash — map seconds and resolution from VideoAgent args.
  44. wan2.2-kf2v-flash — enforce 5s duration.
  45. wan2.7 — document loop seam caveat from picker measurements.







Vidu API implementation checklist

  1. Implement auth header per Vidu docs (API key or signed token — confirm current spec).
  2. Submit text-to-video and image-to-video endpoints as separate profile entries.
  3. Poll task id until success; map vendor states to VideoJob status enum.
  4. Download MP4 from signed URL; SSRF allowlist Vidu CDN hosts only.
  5. Off-peak queue: document optional max_wait_time=172800 for batch jobs.
  6. Expose vidu: job id namespace; strip prefix in get_status/download.
  7. Validation: max prompt length, aspect ratio enums per SKU table in issue body.
  8. Unit tests with httpx mock for submit/poll/download happy path.
  9. Integration test for timeout when poll exceeds max_wait_time.
  10. Catalog sync: static pricing row until public price API exists.
  11. Error mapping: quota exceeded → auth/billing class error.
  12. Example script under examples/python/video/vidu_t2v.py (#10).

DashScope Wan API implementation checklist

  1. Use DashScope Wan video API (KF2V / T2V) per Alibaba documentation.
  2. KF2V: enforce fixed 5s duration — reject other seconds with validation error.
  3. Async task id polling pattern similar to Vidu provider.
  4. Map dashscope: namespace for job ids.
  5. Region endpoint selection via env DASHSCOPE_REGION default cn.
  6. API key from DASHSCOPE_API_KEY; never log.
  7. Handle task failure codes with actionable messages in VideoProviderError.
  8. Download video URL from task result payload.
  9. Mock tests for submit, in-progress, success, failed task JSON fixtures.
  10. Cross-check OpenRouter Wan route vs direct — matrix notes billing difference.
  11. Wire into registry with model prefix dashscope/wan-....
  12. Document in CREDENTIALS.md alongside other Alibaba keys.

Direct vs reseller test matrix

Test id Scenario Fixture Expected
test_direct_wan_01 Scenario 1 mock HTTP pass
test_direct_wan_02 Scenario 2 mock HTTP pass
test_direct_wan_03 Scenario 3 mock HTTP pass
test_direct_wan_04 Scenario 4 mock HTTP pass
test_direct_wan_05 Scenario 5 mock HTTP pass
test_direct_wan_06 Scenario 6 mock HTTP pass
test_direct_wan_07 Scenario 7 mock HTTP pass
test_direct_wan_08 Scenario 8 mock HTTP pass
test_direct_wan_09 Scenario 9 mock HTTP pass
test_direct_wan_10 Scenario 10 mock HTTP pass
test_direct_wan_11 Scenario 11 mock HTTP pass
test_direct_wan_12 Scenario 12 mock HTTP pass
test_direct_wan_13 Scenario 13 mock HTTP pass
test_direct_wan_14 Scenario 14 mock HTTP pass
test_direct_wan_15 Scenario 15 mock HTTP pass
test_direct_wan_16 Scenario 16 mock HTTP pass
test_direct_wan_17 Scenario 17 mock HTTP pass
test_direct_wan_18 Scenario 18 mock HTTP pass
test_direct_wan_19 Scenario 19 mock HTTP pass
test_direct_wan_20 Scenario 20 mock HTTP pass
test_direct_wan_21 Scenario 21 mock HTTP pass
test_direct_wan_22 Scenario 22 mock HTTP pass

Additional verification notes

  • Document Vidu off-peak max_wait_time recommendation: 172800 seconds optional.
  • Map DashScope task failure codes to VideoProviderError messages.
  • Test KF2V rejects seconds=8 with actionable validation text.
  • Document Vidu off-peak max_wait_time recommendation: 172800 seconds optional.
  • Map DashScope task failure codes to VideoProviderError messages.
主要言語
Python
スター
9.3k
フォーク
1.5k
平均マージ
1日 5時間
マージ済み PR(30日)
520

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

MervinPraison/PraisonAI のほかの issue

MervinPraison/PraisonAI の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。