[video] Direct vendor APIs — Vidu and Alibaba DashScope (Wan)

Aperta
#5,176 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
14/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
python
Ambito
ai, api

Direzione di ricerca

The work belongs in praisonaiagents/video/adapters/, where new files vidu.py and dashscope_wan.py are proposed alongside a shared _async_task_poll.py; start by reading the existing video adapters there and how the registry resolves model prefixes. Done means one Vidu route and two DashScope Wan routes (wan2.6-flash and wan2.2-kf2v-flash, which must reject seconds other than 5) pass mocked tests, with DASHSCOPE_API_KEY and VIDU_API_KEY added to .env.example. The issue spans several providers and open design decisions about off-peak job handling, so it needs maintainer scoping first.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug claude documentation

Epic: #5171


Problem statement

The cost picker uses direct vendor APIs to avoid reseller markup:

Vidu Q3 Turbo
  • Channel: Vidu direct (off-peak)
  • 1080p, includes audio
  • Off-peak 48 hour queue vs standard queue ~$0.065/s (picker)
Alibaba Wan (DashScope)
SKU Picker behavior
Wan 2.6 Flash Direct pricing vs OpenRouter ~$0.15/s resold
Wan 2.2 KF2V Flash 5s fixed, silent, ~$0.015/s @ 480p
Wan 2.7 Strong water/leaves motion; loose loop seam on people

PraisonAI TS maps qwen → DASHSCOPE_API_KEY for chat embeddings, not Wan video REST endpoints. No ViduVideoProvider exists.


Vidu API — implementation requirements

Provider id: vidu

Model strings:

VideoAgent(llm="vidu/q3-turbo")
VideoAgent(llm="vidu/q3-turbo-offpeak")  # if separate SKU

Tasks:

  1. Read Vidu official API docs at implementation time (endpoints for text-to-video / image-to-video).

  2. Auth header pattern (likely Authorization: Bearer <API_KEY> — env VIDU_API_KEY).

  3. Queue mode parameter for off-peak vs standard — map kwargs:

    agent.generate(..., queue="offpeak")  # or vendor enum
    
  4. Poll until terminal state; download MP4.

  5. User-facing SLA text: off-peak may take up to 48 hours (picker); do not block wait_for_completion default 600s — require max_wait_time override or async job pattern:

    Design decision: For off-peak, generate() returns job id immediately; document that user must poll with long max_wait_time or use webhooks if Vidu supports them.

Error codes: map 402/429/quota to clear messages.


Alibaba DashScope Wan — implementation requirements

Provider id: dashscope (preferred prefix for consistency with Qwen)

Model strings (illustrative — pin exact API model ids in PR):

VideoAgent(llm="dashscope/wan2.6-flash")
VideoAgent(llm="dashscope/wan2.2-kf2v-flash")
VideoAgent(llm="dashscope/wan2.7")

Environment:

  • DASHSCOPE_API_KEY (same as Qwen)

API pattern (typical for DashScope async tasks):

  1. POST create async task with model, input, parameters
  2. Poll GET task status
  3. Retrieve output.video_url

Implement against current Alibaba documentation for Wan video (not text generation).

KF2V Flash rules:

  • If user passes seconds != "5", raise VideoProviderValidationError or clamp with warning (picker: fixed 5s).
  • Silent output — no audio track in deliverable.

Resolution mapping:

Picker Parameter
480p resolution="480P" or equivalent
720p / 1080p map from size or kwargs

Channel selection (document in matrix #10)

For Wan 2.7 same picker row may list OpenRouter + Alibaba direct:

llm= prefix Channel
dashscope/wan2.7 Direct (lower $/s per picker)
openrouter/... Single key multi-model (#03)
replicate/... Flat-run variants (#04)

Registry must treat these as different providers with different job id namespaces.


ViduVideoProvider / DashScopeWanVideoProvider files

praisonaiagents/video/adapters/vidu.py
praisonaiagents/video/adapters/dashscope_wan.py

Shared utilities:

  • praisonaiagents/video/adapters/_async_task_poll.py — generic poll loop with backoff if both vendors share pattern

Testing

Vidu mocks:

  • off-peak submit returns status=queued
  • standard queue faster completion

DashScope mocks:

  • KF2V rejects wrong duration
  • 2.6 Flash success path

No live keys in CI.


Examples

  • examples/python/video/video_vidu_q3_turbo_offpeak.py
  • examples/python/video/video_dashscope_wan_kf2v.py
  • examples/python/video/video_dashscope_wan27.py

Acceptance criteria

  • One Vidu + two Alibaba routes working in mocked tests
  • Off-peak documented with long-running job guidance
  • Env vars in .env.example
  • Estimator direct $/s from catalog (#08) differs from OpenRouter same SKU

Dependencies

Requires #02

Related

Epic #01, #03, #04, #08, #10


Appendix: Vidu off-peak job UX

  • Return job id immediately; document expected queue latency up to 48h.
  • Provide estimate_queue_wait() in catalog, not in provider.
  • Desktop (#09) should show 'queued' state without blocking UI thread for 48h.

Appendix: DashScope Wan SKU checklist

  1. wan2.6-flash — map seconds and resolution from VideoAgent args.
  2. wan2.2-kf2v-flash — enforce 5s duration.
  3. wan2.7 — document loop seam caveat from picker measurements.
  4. wan2.6-flash — map seconds and resolution from VideoAgent args.
  5. wan2.2-kf2v-flash — enforce 5s duration.
  6. wan2.7 — document loop seam caveat from picker measurements.
  7. wan2.6-flash — map seconds and resolution from VideoAgent args.
  8. wan2.2-kf2v-flash — enforce 5s duration.
  9. wan2.7 — document loop seam caveat from picker measurements.
  10. wan2.6-flash — map seconds and resolution from VideoAgent args.
  11. wan2.2-kf2v-flash — enforce 5s duration.
  12. wan2.7 — document loop seam caveat from picker measurements.
  13. wan2.6-flash — map seconds and resolution from VideoAgent args.
  14. wan2.2-kf2v-flash — enforce 5s duration.
  15. wan2.7 — document loop seam caveat from picker measurements.
  16. wan2.6-flash — map seconds and resolution from VideoAgent args.
  17. wan2.2-kf2v-flash — enforce 5s duration.
  18. wan2.7 — document loop seam caveat from picker measurements.
  19. wan2.6-flash — map seconds and resolution from VideoAgent args.
  20. wan2.2-kf2v-flash — enforce 5s duration.
  21. wan2.7 — document loop seam caveat from picker measurements.
  22. wan2.6-flash — map seconds and resolution from VideoAgent args.
  23. wan2.2-kf2v-flash — enforce 5s duration.
  24. wan2.7 — document loop seam caveat from picker measurements.
  25. wan2.6-flash — map seconds and resolution from VideoAgent args.
  26. wan2.2-kf2v-flash — enforce 5s duration.
  27. wan2.7 — document loop seam caveat from picker measurements.
  28. wan2.6-flash — map seconds and resolution from VideoAgent args.
  29. wan2.2-kf2v-flash — enforce 5s duration.
  30. wan2.7 — document loop seam caveat from picker measurements.
  31. wan2.6-flash — map seconds and resolution from VideoAgent args.
  32. wan2.2-kf2v-flash — enforce 5s duration.
  33. wan2.7 — document loop seam caveat from picker measurements.
  34. wan2.6-flash — map seconds and resolution from VideoAgent args.
  35. wan2.2-kf2v-flash — enforce 5s duration.
  36. wan2.7 — document loop seam caveat from picker measurements.
  37. wan2.6-flash — map seconds and resolution from VideoAgent args.
  38. wan2.2-kf2v-flash — enforce 5s duration.
  39. wan2.7 — document loop seam caveat from picker measurements.
  40. wan2.6-flash — map seconds and resolution from VideoAgent args.
  41. wan2.2-kf2v-flash — enforce 5s duration.
  42. wan2.7 — document loop seam caveat from picker measurements.
  43. wan2.6-flash — map seconds and resolution from VideoAgent args.
  44. wan2.2-kf2v-flash — enforce 5s duration.
  45. wan2.7 — document loop seam caveat from picker measurements.







Vidu API implementation checklist

  1. Implement auth header per Vidu docs (API key or signed token — confirm current spec).
  2. Submit text-to-video and image-to-video endpoints as separate profile entries.
  3. Poll task id until success; map vendor states to VideoJob status enum.
  4. Download MP4 from signed URL; SSRF allowlist Vidu CDN hosts only.
  5. Off-peak queue: document optional max_wait_time=172800 for batch jobs.
  6. Expose vidu: job id namespace; strip prefix in get_status/download.
  7. Validation: max prompt length, aspect ratio enums per SKU table in issue body.
  8. Unit tests with httpx mock for submit/poll/download happy path.
  9. Integration test for timeout when poll exceeds max_wait_time.
  10. Catalog sync: static pricing row until public price API exists.
  11. Error mapping: quota exceeded → auth/billing class error.
  12. Example script under examples/python/video/vidu_t2v.py (#10).

DashScope Wan API implementation checklist

  1. Use DashScope Wan video API (KF2V / T2V) per Alibaba documentation.
  2. KF2V: enforce fixed 5s duration — reject other seconds with validation error.
  3. Async task id polling pattern similar to Vidu provider.
  4. Map dashscope: namespace for job ids.
  5. Region endpoint selection via env DASHSCOPE_REGION default cn.
  6. API key from DASHSCOPE_API_KEY; never log.
  7. Handle task failure codes with actionable messages in VideoProviderError.
  8. Download video URL from task result payload.
  9. Mock tests for submit, in-progress, success, failed task JSON fixtures.
  10. Cross-check OpenRouter Wan route vs direct — matrix notes billing difference.
  11. Wire into registry with model prefix dashscope/wan-....
  12. Document in CREDENTIALS.md alongside other Alibaba keys.

Direct vs reseller test matrix

Test id Scenario Fixture Expected
test_direct_wan_01 Scenario 1 mock HTTP pass
test_direct_wan_02 Scenario 2 mock HTTP pass
test_direct_wan_03 Scenario 3 mock HTTP pass
test_direct_wan_04 Scenario 4 mock HTTP pass
test_direct_wan_05 Scenario 5 mock HTTP pass
test_direct_wan_06 Scenario 6 mock HTTP pass
test_direct_wan_07 Scenario 7 mock HTTP pass
test_direct_wan_08 Scenario 8 mock HTTP pass
test_direct_wan_09 Scenario 9 mock HTTP pass
test_direct_wan_10 Scenario 10 mock HTTP pass
test_direct_wan_11 Scenario 11 mock HTTP pass
test_direct_wan_12 Scenario 12 mock HTTP pass
test_direct_wan_13 Scenario 13 mock HTTP pass
test_direct_wan_14 Scenario 14 mock HTTP pass
test_direct_wan_15 Scenario 15 mock HTTP pass
test_direct_wan_16 Scenario 16 mock HTTP pass
test_direct_wan_17 Scenario 17 mock HTTP pass
test_direct_wan_18 Scenario 18 mock HTTP pass
test_direct_wan_19 Scenario 19 mock HTTP pass
test_direct_wan_20 Scenario 20 mock HTTP pass
test_direct_wan_21 Scenario 21 mock HTTP pass
test_direct_wan_22 Scenario 22 mock HTTP pass

Additional verification notes

  • Document Vidu off-peak max_wait_time recommendation: 172800 seconds optional.
  • Map DashScope task failure codes to VideoProviderError messages.
  • Test KF2V rejects seconds=8 with actionable validation text.
  • Document Vidu off-peak max_wait_time recommendation: 172800 seconds optional.
  • Map DashScope task failure codes to VideoProviderError messages.
Lingua principale
Python
Stelle
9.3k
Fork
1.5k
Merge medio
1g 5h
PR unite (30g)
520

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di MervinPraison/PraisonAI

Tutte le issue di MervinPraison/PraisonAI

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.