Integration harness marks contract failures PASS; JS SDK npm timeout
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 初心者へのやさしさ
- 45/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- javascript, python
調査の方向性
Start with the live SDK E2E script at e2e-audit-logs/praisonai-live-sdk-e2e-origin-main-20260826.py, specifically run_case, where ok is set to True whenever no exception is raised, and the summary that ignores notes.contract_ok. Make the case fail when contract_ok is False or a subprocess returncode is non-zero, then confirm that the structured, memory, guardrail and yaml CLI cases report FAIL. Done means the summary shows those cases as failed and a unit test in src/praisonai-agents/tests/ covers the harness rule. The js_sdk_hello npm timeout is a separate question and should be scoped separately.
索引モデルが issue の本文から書いたものです。
説明
[Bug] Integration harness marks contract failures PASS; JS SDK npm timeout
Metadata
| Field | Value |
|---|---|
| Repository | MervinPraison/PraisonAI |
| Release tested | v4.7.11 |
| Commit (tag) | 08079647275c500f27e02ce7fa229f15b0308f8f |
| Labels | bug, testing, e2e, dx, v4.7.11 |
| Priority | P2 |
Executive summary
The live SDK E2E script reports 15/18 PASS on v4.7.11 while structured, memory, guardrail, yaml CLI, and RAG cases violate documented contracts. Additionally js_sdk_hello fails on npm install 240s timeout. Harness must fail closed.
Interactive triage
flowchart TD
A[Live E2E case marked PASS?] -->|No| Z
A -->|Yes| B{contract_ok or returncode?}
B -->|false / non-zero| C[Harness should fail case]
B -->|true| OK
Two classes: silent contract failures (010/011/012) and js_sdk_hello npm timeout.
Terminal — summary v4.7.11 run
=== SUMMARY ===
passed=15 failed=3
failed_names= ['mcp_time_server', 'mcp_npx_inspect', 'js_sdk_hello']
silent_contract= [structured output, memory recall, guardrails — contract_ok false but case PASS]
SHA=08079647275c500f27e02ce7fa229f15b0308f8f
Harness vs honest score
| Metric | Count |
|---|---|
| Harness pass | 15 |
| Harness fail | 3 |
| Should-fail (contract) | 3+ (structured, memory, guardrail, yaml run, rag kwarg) |
| Honest effective | ~12–13 / 18 |
js_sdk_hello failure
[FAIL] js_sdk_hello 249.78s
TimeoutExpired: npm install --no-audit --no-fund timed out after 240 seconds
| Mitigation | |
|---|---|
| Pre-install deps in CI | npm ci cache |
| Skip if no node_modules | Use existing dist/ |
| Increase timeout | Dev laptops slow disk |
Proposed harness rules
| Rule | Apply to |
|---|---|
Fail if contract_ok is False |
structured, memory |
Fail if returncode != 0 |
yaml_cli_run, rag subprocess |
Fail if is_none on guardrail case |
guardrail_blocks |
Full validation report citation
Release QA on v4.7.11 found multiple SDK contract gaps; integration harness should fail closed when contract_ok is false.
Harness rule changes
- HR-001: After each case, assert notes.contract_ok is not False.
- HR-002: yaml_cli_run: require returncode==0 and READY in output.
- HR-003: rag_knowledge_docs: require returncode==0.
- HR-004: guardrail_blocks: fail if is_none.
JS SDK case
| Issue | Mitigation |
|---|---|
| npm install timeout | Prebuilt node_modules in CI; timeout 600s |
| Cold network | Skip with SKIP_JS_E2E=1 locally |
Test matrix
| ID | Scenario | Expected |
|---|---|---|
| HAR-001 | Inject contract_ok false → case fails | Pass on fixed build |
| HAR-002 | Inject returncode 1 yaml → case fails | Pass on fixed build |
| HAR-003 | js_sdk with npm ci cache → pass | Pass on fixed build |
Files to inspect
| Path | Why |
|---|---|
e2e-audit-logs/praisonai-live-sdk-e2e-origin-main-20260826.py |
Primary suspect |
Root cause hypotheses
- RC-001: run_case marks ok=True whenever no exception
- RC-002: contract_ok not consulted in summary
- RC-003: npm install not cached for praisonai-ts
Integration scenarios (extended)
| ID | Scenario | Expected |
|---|---|---|
| INT-001 | Clean Windows VM with tag checkout only | Pass on fixed build |
| INT-002 | Repeat with empty praisonai config.yaml | Pass on fixed build |
| INT-003 | Repeat with OPENAI_API_KEY from CI secret | Pass on fixed build |
| INT-004 | Compare behavior on current main branch | Pass on fixed build |
| INT-005 | PyPI wheel install vs source worktree | Pass on fixed build |
| INT-006 | Regression after fix on adjacent Agent APIs | Pass on fixed build |
Validation context
| Item | Detail |
|---|---|
| Release tested | v4.7.11 |
| Git commit (tag) | 08079647275c500f27e02ce7fa229f15b0308f8f |
| When | 2026-10-02 |
| OS | Windows 10/11 |
| Python | 3.13 |
| Model | gpt-4o-mini (live OpenAI) |
| Integration case | js_sdk_hello + contract cases / js_sdk_hello |
Environment
| Dependency | Notes |
|---|---|
OPENAI_API_KEY |
Required for live repro |
| Monorepo install | pip install -e src/praisonai-agents (+ wrapper if using CLI) |
Reproduction runbook
- Check out tag v4.7.11 in the PraisonAI monorepo.
- Install editable packages from
src/praisonai-agents(andsrc/praisonaifor CLI bugs). - Set
OPENAI_API_KEYin the environment. - Run the minimal repro in the Terminal evidence section above.
- For regression, add or extend tests under
src/praisonai-agents/tests/. - Open a PR with fix + test; link this GitHub issue.
Non-functional requirements
- NFR-001: Fix must not regress basic Agent.chat smoke on gpt-4o-mini.
- NFR-002: No new mandatory env vars for default Agent.chat path.
- NFR-003: New unit tests complete in under 30s locally.
Related defects (same release QA pass)
| Area | Symptom |
|---|---|
| Structured output | output_pydantic returns str |
| Memory | user_id not recalled across sessions |
| Guardrails | Blocked run returns None |
| CLI YAML | maximum recursion depth exceeded on run |
| MCP | fileno init error on Windows |
| RAG API | retrieval_config rejected on Agent |
| Test harness | contract_ok: false still reported PASS |
Security
- SEC-001: Do not paste API keys into issues or CI logs.
- SEC-002: Memory fixes must preserve isolation between
user_idvalues.
Open questions
- OQ-001: Publish harness as pytest plugin for CI?
- OQ-002: Separate quick vs full E2E profiles?
Assignee checklist
- CHK-001: Reproduce on release tag v4.7.11 before coding.
- CHK-002: Link PR to this GitHub issue.
- CHK-003: Add unit test under
src/praisonai-agents/tests/. - CHK-004: Update integration harness to fail when contract breaks (structured/memory/guardrail).
- CHK-005: Update Mintlify if user-facing behavior changes.
Acceptance criteria
- Reproduction steps fail on v4.7.11 and pass on fix PR.
- Unit or integration test in
src/praisonai-agents/tests/. - Documented in Mintlify if user-facing API change.
Regression checks
| Check | How |
|---|---|
| Unit | pytest src/praisonai-agents/tests/ -k '<relevant>' |
| Live smoke | Minimal repro below with OPENAI_API_KEY set |
| Focus | Assert contract_ok and non-zero exit codes in live integration harness |
After fix: confirm behavior on main and plan cherry-pick to next patch release if needed.
- 主要言語
- Python
- スター
- 9.3k
- フォーク
- 1.5k
- 平均マージ
- 1日 5時間
- マージ済み PR(30日)
- 520
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
MervinPraison/PraisonAI のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
MervinPraison/PraisonAI#6710 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
claude
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
MervinPraison/PraisonAI#6707 ·
メンテナーはふだん 1 日以内に返信
-
bug claude
難易度 1/5 1時間未満 初心者へのやさしさ 90/100
MervinPraison/PraisonAI#6671 ·
メンテナーはふだん 1 日以内に返信
-
Compaction budget regression fixture remains below budget with accurate tokenization再び着手できるかも このイシューのプルリクエストはマージされずにクローズされました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 62/100
MervinPraison/PraisonAI#5418 · コメント 3 件 ·
メンテナーはふだん 1 日以内に返信
-
bug claude documentation
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
MervinPraison/PraisonAI#5341 · コメント 3 件 ·
メンテナーはふだん 1 日以内に返信
MervinPraison/PraisonAI の issue をすべて見る
似ている issue
-
cognition
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 78/100
collabnix/awesome-mcp-lists#433 ·
メンテナーはふだん 4 日以内に返信
-
CrossEntropyFilter raises on zero probabilities and single-class results対応中かも @NopeCharon が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
datajoint/datajoint-python#1703 ·
メンテナーはふだん 6 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
Archmonger/ServeStatic#129 ·