Integration harness marks contract failures PASS; JS SDK npm timeout
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 45/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Tranquilla
- Stack tecnologico
- javascript, python
Direzione di ricerca
Start with the live SDK E2E script at e2e-audit-logs/praisonai-live-sdk-e2e-origin-main-20260826.py, specifically run_case, where ok is set to True whenever no exception is raised, and the summary that ignores notes.contract_ok. Make the case fail when contract_ok is False or a subprocess returncode is non-zero, then confirm that the structured, memory, guardrail and yaml CLI cases report FAIL. Done means the summary shows those cases as failed and a unit test in src/praisonai-agents/tests/ covers the harness rule. The js_sdk_hello npm timeout is a separate question and should be scoped separately.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
[Bug] Integration harness marks contract failures PASS; JS SDK npm timeout
Metadata
| Field | Value |
|---|---|
| Repository | MervinPraison/PraisonAI |
| Release tested | v4.7.11 |
| Commit (tag) | 08079647275c500f27e02ce7fa229f15b0308f8f |
| Labels | bug, testing, e2e, dx, v4.7.11 |
| Priority | P2 |
Executive summary
The live SDK E2E script reports 15/18 PASS on v4.7.11 while structured, memory, guardrail, yaml CLI, and RAG cases violate documented contracts. Additionally js_sdk_hello fails on npm install 240s timeout. Harness must fail closed.
Interactive triage
flowchart TD
A[Live E2E case marked PASS?] -->|No| Z
A -->|Yes| B{contract_ok or returncode?}
B -->|false / non-zero| C[Harness should fail case]
B -->|true| OK
Two classes: silent contract failures (010/011/012) and js_sdk_hello npm timeout.
Terminal — summary v4.7.11 run
=== SUMMARY ===
passed=15 failed=3
failed_names= ['mcp_time_server', 'mcp_npx_inspect', 'js_sdk_hello']
silent_contract= [structured output, memory recall, guardrails — contract_ok false but case PASS]
SHA=08079647275c500f27e02ce7fa229f15b0308f8f
Harness vs honest score
| Metric | Count |
|---|---|
| Harness pass | 15 |
| Harness fail | 3 |
| Should-fail (contract) | 3+ (structured, memory, guardrail, yaml run, rag kwarg) |
| Honest effective | ~12–13 / 18 |
js_sdk_hello failure
[FAIL] js_sdk_hello 249.78s
TimeoutExpired: npm install --no-audit --no-fund timed out after 240 seconds
| Mitigation | |
|---|---|
| Pre-install deps in CI | npm ci cache |
| Skip if no node_modules | Use existing dist/ |
| Increase timeout | Dev laptops slow disk |
Proposed harness rules
| Rule | Apply to |
|---|---|
Fail if contract_ok is False |
structured, memory |
Fail if returncode != 0 |
yaml_cli_run, rag subprocess |
Fail if is_none on guardrail case |
guardrail_blocks |
Full validation report citation
Release QA on v4.7.11 found multiple SDK contract gaps; integration harness should fail closed when contract_ok is false.
Harness rule changes
- HR-001: After each case, assert notes.contract_ok is not False.
- HR-002: yaml_cli_run: require returncode==0 and READY in output.
- HR-003: rag_knowledge_docs: require returncode==0.
- HR-004: guardrail_blocks: fail if is_none.
JS SDK case
| Issue | Mitigation |
|---|---|
| npm install timeout | Prebuilt node_modules in CI; timeout 600s |
| Cold network | Skip with SKIP_JS_E2E=1 locally |
Test matrix
| ID | Scenario | Expected |
|---|---|---|
| HAR-001 | Inject contract_ok false → case fails | Pass on fixed build |
| HAR-002 | Inject returncode 1 yaml → case fails | Pass on fixed build |
| HAR-003 | js_sdk with npm ci cache → pass | Pass on fixed build |
Files to inspect
| Path | Why |
|---|---|
e2e-audit-logs/praisonai-live-sdk-e2e-origin-main-20260826.py |
Primary suspect |
Root cause hypotheses
- RC-001: run_case marks ok=True whenever no exception
- RC-002: contract_ok not consulted in summary
- RC-003: npm install not cached for praisonai-ts
Integration scenarios (extended)
| ID | Scenario | Expected |
|---|---|---|
| INT-001 | Clean Windows VM with tag checkout only | Pass on fixed build |
| INT-002 | Repeat with empty praisonai config.yaml | Pass on fixed build |
| INT-003 | Repeat with OPENAI_API_KEY from CI secret | Pass on fixed build |
| INT-004 | Compare behavior on current main branch | Pass on fixed build |
| INT-005 | PyPI wheel install vs source worktree | Pass on fixed build |
| INT-006 | Regression after fix on adjacent Agent APIs | Pass on fixed build |
Validation context
| Item | Detail |
|---|---|
| Release tested | v4.7.11 |
| Git commit (tag) | 08079647275c500f27e02ce7fa229f15b0308f8f |
| When | 2026-10-02 |
| OS | Windows 10/11 |
| Python | 3.13 |
| Model | gpt-4o-mini (live OpenAI) |
| Integration case | js_sdk_hello + contract cases / js_sdk_hello |
Environment
| Dependency | Notes |
|---|---|
OPENAI_API_KEY |
Required for live repro |
| Monorepo install | pip install -e src/praisonai-agents (+ wrapper if using CLI) |
Reproduction runbook
- Check out tag v4.7.11 in the PraisonAI monorepo.
- Install editable packages from
src/praisonai-agents(andsrc/praisonaifor CLI bugs). - Set
OPENAI_API_KEYin the environment. - Run the minimal repro in the Terminal evidence section above.
- For regression, add or extend tests under
src/praisonai-agents/tests/. - Open a PR with fix + test; link this GitHub issue.
Non-functional requirements
- NFR-001: Fix must not regress basic Agent.chat smoke on gpt-4o-mini.
- NFR-002: No new mandatory env vars for default Agent.chat path.
- NFR-003: New unit tests complete in under 30s locally.
Related defects (same release QA pass)
| Area | Symptom |
|---|---|
| Structured output | output_pydantic returns str |
| Memory | user_id not recalled across sessions |
| Guardrails | Blocked run returns None |
| CLI YAML | maximum recursion depth exceeded on run |
| MCP | fileno init error on Windows |
| RAG API | retrieval_config rejected on Agent |
| Test harness | contract_ok: false still reported PASS |
Security
- SEC-001: Do not paste API keys into issues or CI logs.
- SEC-002: Memory fixes must preserve isolation between
user_idvalues.
Open questions
- OQ-001: Publish harness as pytest plugin for CI?
- OQ-002: Separate quick vs full E2E profiles?
Assignee checklist
- CHK-001: Reproduce on release tag v4.7.11 before coding.
- CHK-002: Link PR to this GitHub issue.
- CHK-003: Add unit test under
src/praisonai-agents/tests/. - CHK-004: Update integration harness to fail when contract breaks (structured/memory/guardrail).
- CHK-005: Update Mintlify if user-facing behavior changes.
Acceptance criteria
- Reproduction steps fail on v4.7.11 and pass on fix PR.
- Unit or integration test in
src/praisonai-agents/tests/. - Documented in Mintlify if user-facing API change.
Regression checks
| Check | How |
|---|---|
| Unit | pytest src/praisonai-agents/tests/ -k '<relevant>' |
| Live smoke | Minimal repro below with OPENAI_API_KEY set |
| Focus | Assert contract_ok and non-zero exit codes in live integration harness |
After fix: confirm behavior on main and plan cherry-pick to next patch release if needed.
- Lingua principale
- Python
- Stelle
- 9.3k
- Fork
- 1.5k
- Merge medio
- 1g 5h
- PR unite (30g)
- 520
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di MervinPraison/PraisonAI
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
MervinPraison/PraisonAI#6710 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
claude
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
MervinPraison/PraisonAI#6707 ·
I maintainer di solito rispondono entro 1 giorno
-
bug claude
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
MervinPraison/PraisonAI#6671 ·
I maintainer di solito rispondono entro 1 giorno
-
Compaction budget regression fixture remains below budget with accurate tokenizationForse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
MervinPraison/PraisonAI#5418 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
bug claude documentation
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
MervinPraison/PraisonAI#5341 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di MervinPraison/PraisonAI
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 2 giorni
-
[Bug]: LANGSMITH_SAMPLING_RATE ignores fractional values (isdigit check), traces 100% of requestsAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
I maintainer di solito rispondono entro 1 giorno
-
platform/backend tests
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
Significant-Gravitas/AutoGPT#17100 ·
I maintainer di solito rispondono entro 1 giorno
-
bug good first issue
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 61/100
scverse/infercnvpy#238 ·
I maintainer di solito rispondono entro 1 giorno