[video] fal.ai video generation

オープン
#5,175 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
22/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
静か
技術スタック
python
領域
ai

調査の方向性

Start with the VideoProvider interface and the TypeScript fal entry in src/praisonai-ts/src/llm/providers/ai-sdk/types.ts, then design the new praisonaiagents/video/adapters/fal.py and fal_models.yaml. Several endpoints are still marked TBD, so verify them on fal.ai first. Done means the three fal routes submit, poll and download through mocked queue responses, and LTX rejects durations under 6 seconds.

索引モデルが issue の本文から書いたものです。

説明

bug claude documentation javascript performance

Epic: #5171


Problem statement

fal.ai routes in the Video Model Cost Picker:

Route Picker detail
MiniMax H3 Max (free tier) 5 gens / 24h @ 480p/768p; verify API vs web-only free tier
LTX-2.3 Fast CHEAP; minimum 6 seconds
Gemini Omni Flash 1.1 Also on Gemini API; first+last frame; TOP RANKED

PraisonAI TypeScript provider registry (src/praisonai-ts/src/llm/providers/ai-sdk/types.ts):

fal: { package: '@ai-sdk/fal', envKey: 'FAL_KEY', modalities: IMAGE_GEN, ... }
luma: { ..., description: 'Luma AI video generation' }  // not wired to Python VideoAgent

Python VideoAgent has zero fal video code.


fal.ai integration pattern

fal models typically use queue API:

  1. POST https://queue.fal.run/{model_id} or REST from model page
  2. Receive request_id
  3. Poll GET .../requests/{request_id}/status or use client SDK
  4. Read response.video.url (field names vary by model)

Do not assume one JSON schema. Implement endpoint map:

# praisonaiagents/video/fal_models.yaml
ltx-2.3-fast:
  endpoint: "fal-ai/ltx-video-2.3-fast"  # verify on fal.ai at implement time
  min_duration: 6
  default_resolution: "720p"
minimax-h3-max:
  endpoint: "..."
  experimental_free_tier: true
gemini-omni-flash-11:
  endpoint: "..."
  supports_first_last_frame: true

Authentication

Authorization: Key $FAL_KEY

Or header format per fal docs (Authorization: Bearer ... — confirm current fal spec in PR).

Env: FAL_KEY (match TS envKey).


FalVideoProvider implementation

File: praisonaiagents/video/adapters/fal.py

class FalVideoProvider(VideoProvider):
    provider_id = "fal"

    def generate(...) -> VideoJob:
        model_id = self._strip_prefix(model)  # fal/ltx-2.3-fast -> config key
        cfg = self._load_model_config(model_id)
        self._validate_duration(seconds, cfg.min_duration)
        payload = self._build_payload(prompt, seconds, size, input_reference, kwargs, cfg)
        request_id = self._submit(cfg.endpoint, payload)
        return VideoJob(id=f"fal:{request_id}", status="pending", ...)

    def get_status(...): ...
    def download(...): ...
LTX-2.3 Fast validation
if int(seconds) < 6:
    raise VideoProviderValidationError(
        "LTX-2.3 Fast requires duration >= 6 seconds (cost picker minimum)"
    )
MiniMax H3 Max
  • Example README blockquote: verify whether API calls consume free tier
  • Optional env gate: FAL_CONFIRM_PAID_USAGE=1 before submit
Gemini Omni Flash 1.1 (fal path)

Map:

  • input_reference → first frame image (URL/base64/upload)
  • last_frame kwarg → second frame
  • Document overlap with native Gemini video APIs to avoid double documentation (#10)

Model naming

VideoAgent(llm="fal/ltx-2.3-fast")
VideoAgent(llm="fal/minimax-h3-max")
VideoAgent(llm="fal/gemini-omni-flash-1.1")

Registry prefix fal/.


Async / long-running jobs

fal jobs may exceed VideoConfig.timeout (default 600s). Allow:

  • VideoConfig.max_wait_time up to 3600 for slow queues
  • Progress logging in wait_for_completion

Testing

Mock queue responses:

  1. submit → { "request_id": "..." }
  2. status IN_PROGRESS → COMPLETED
  3. completed payload with { "video": { "url": "https://..." } }

Tests:

  • LTX rejects seconds=5
  • Omni builds two-frame payload
  • Missing FAL_KEY → auth error

Examples

  • examples/python/video/video_fal_ltx23.py — duration=6, resolution from picker
  • examples/python/video/video_fal_minimax_h3.py — billing warning header
  • examples/python/video/video_fal_omni_flash.py — two-frame I2V

Acceptance criteria

  • Three fal picker routes implemented (config-driven)
  • fal YAML config validated in CI (schema check)
  • Registry + CLI + MCP integration
  • Matrix rows with fal channel marked Supported

Dependencies

Requires #02

Related

Epic #01, #08, #10

References


Appendix: fal model config schema

models:
  ltx-2.3-fast:
    endpoint: fal-ai/ltx-video-2.3-fast
    min_duration: 6
  minimax-h3-max:
    endpoint: TBD
    free_tier_unverified: true
  gemini-omni-flash-1.1:
    endpoint: TBD
    two_frame: true

Appendix: Validation rules

  1. Rule 1: validate payload keys against YAML before HTTP
  2. Rule 2: validate payload keys against YAML before HTTP
  3. Rule 3: validate payload keys against YAML before HTTP
  4. Rule 4: validate payload keys against YAML before HTTP
  5. Rule 5: validate payload keys against YAML before HTTP
  6. Rule 6: validate payload keys against YAML before HTTP
  7. Rule 7: validate payload keys against YAML before HTTP
  8. Rule 8: validate payload keys against YAML before HTTP
  9. Rule 9: validate payload keys against YAML before HTTP
  10. Rule 10: validate payload keys against YAML before HTTP
  11. Rule 11: validate payload keys against YAML before HTTP
  12. Rule 12: validate payload keys against YAML before HTTP
  13. Rule 13: validate payload keys against YAML before HTTP
  14. Rule 14: validate payload keys against YAML before HTTP
  15. Rule 15: validate payload keys against YAML before HTTP
  16. Rule 16: validate payload keys against YAML before HTTP
  17. Rule 17: validate payload keys against YAML before HTTP
  18. Rule 18: validate payload keys against YAML before HTTP
  19. Rule 19: validate payload keys against YAML before HTTP
  20. Rule 20: validate payload keys against YAML before HTTP
  21. Rule 21: validate payload keys against YAML before HTTP
  22. Rule 22: validate payload keys against YAML before HTTP
  23. Rule 23: validate payload keys against YAML before HTTP
  24. Rule 24: validate payload keys against YAML before HTTP
  25. Rule 25: validate payload keys against YAML before HTTP
  26. Rule 26: validate payload keys against YAML before HTTP
  27. Rule 27: validate payload keys against YAML before HTTP
  28. Rule 28: validate payload keys against YAML before HTTP







fal queue lifecycle

  1. POST queue endpoint with model id and input dict (see fal_models.yaml profile).
  2. Receive request_id and queue position; namespaced job id fal:{request_id}.
  3. Poll status URL until COMPLETED or FAILED (map to internal terminal set).
  4. Download result URL from response; validate host against fal.media allowlist (#02 SSRF).
  5. LTX profiles: enforce minimum 6s duration in validator before submit.
  6. Omni two-frame: require input_reference; last_frame optional per profile.
  7. Handle 422 validation errors — surface fal message text to agent user.
  8. Async path mirrors sync with httpx.AsyncClient for agenerate.

fal integration tests (enumeration)

Test id Scenario Fixture Expected
test_fal_01 Scenario 1 mock HTTP pass
test_fal_02 Scenario 2 mock HTTP pass
test_fal_03 Scenario 3 mock HTTP pass
test_fal_04 Scenario 4 mock HTTP pass
test_fal_05 Scenario 5 mock HTTP pass
test_fal_06 Scenario 6 mock HTTP pass
test_fal_07 Scenario 7 mock HTTP pass
test_fal_08 Scenario 8 mock HTTP pass
test_fal_09 Scenario 9 mock HTTP pass
test_fal_10 Scenario 10 mock HTTP pass
test_fal_11 Scenario 11 mock HTTP pass
test_fal_12 Scenario 12 mock HTTP pass
test_fal_13 Scenario 13 mock HTTP pass
test_fal_14 Scenario 14 mock HTTP pass
test_fal_15 Scenario 15 mock HTTP pass
test_fal_16 Scenario 16 mock HTTP pass
test_fal_17 Scenario 17 mock HTTP pass
test_fal_18 Scenario 18 mock HTTP pass
test_fal_19 Scenario 19 mock HTTP pass
test_fal_20 Scenario 20 mock HTTP pass
test_fal_21 Scenario 21 mock HTTP pass
test_fal_22 Scenario 22 mock HTTP pass
test_fal_23 Scenario 23 mock HTTP pass
test_fal_24 Scenario 24 mock HTTP pass
test_fal_25 Scenario 25 mock HTTP pass
test_fal_26 Scenario 26 mock HTTP pass
test_fal_27 Scenario 27 mock HTTP pass
test_fal_28 Scenario 28 mock HTTP pass

Billing verification checklist

  1. Confirm FAL_KEY scoped to project with video models enabled.
  2. Catalog row pricing.mode=per_second matches fal published rates within 1%.
  3. Log usage or billed seconds when API returns them for #08 reconciliation.
  4. Alert if daily spend exceeds threshold (external — document hook point in provider).
  5. Test refund/failure path — failed jobs must not increment success metrics.
  6. Document that fal queue time is not billable unless provider charges queue fee.
  7. Compare estimator vs one live 5s LTX clip monthly in staging.
  8. Ensure failed downloads do not retry infinitely (cap at 3).
  9. Verify USD display uses Decimal in #08 not float accumulation.
  10. Add fal model id to SUPPORT_MATRIX with implementation_status=supported.

Additional verification notes

主要言語
Python
スター
9.3k
フォーク
1.5k
平均マージ
1日 5時間
マージ済み PR(30日)
520

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

MervinPraison/PraisonAI のほかの issue

MervinPraison/PraisonAI の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。