[Enhancement] Durable workflow suspend and resume with persisted execution state (PAI-TS-001)

Abierto
#5,429 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
12/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
sqlite, typescript

Línea de trabajo

Start in src/praisonai-ts/src/workflows/index.ts to see how AgentFlow steps run and where WorkflowContext lives in memory, then read src/praisonai-ts/src/db/ for the existing SQLite session patterns. The design calls for a new WorkflowStorageAdapter with suspend and resume, so this is design work rather than a first task. Done means the WF-SUSP and WF-UNIT tests in the issue pass against a real adapter.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug claude documentation javascript security

[Enhancement] Durable workflow suspend and resume with persisted execution state

Metadata

Field Value
Repository MervinPraison/PraisonAI
Package praisonai (npm) / monorepo path src/praisonai-ts
Component workflows, session storage, approval
Labels enhancement, workflows, human-in-the-loop, storage, praisonai-ts
Priority P1
Estimated effort 3–5 engineer-weeks

Executive summary

Production agent workflows often require pausing execution until a human approves an action, until external data arrives, or until a compliance review completes. The TypeScript SDK supports workflow orchestration, branching, parallelism, and YAML-defined flows, but it does not yet provide a first-class, storage-backed suspend and resume contract at the workflow step level. Tool-level restart safety exists; approval modules exist; however, there is no durable checkpoint that survives process restarts and no canonical API for operators to resume a workflow run days later.

This issue defines the product requirement, public API, persistence model, security constraints, and exhaustive test matrix needed to ship durable suspend/resume for AgentFlow and related workflow patterns.

Interactive triage (start here)

Use this decision tree to confirm the gap applies to your report:

flowchart TD
  A[Workflow needs human approval mid-run?] -->|Yes| B[Process may restart before approval?]
  A -->|No| Z[Close as not applicable]
  B -->|Yes| C{After restart, can you resume same run id?}
  B -->|No| Z
  C -->|No| D[File under this issue]
  C -->|Yes| Z
  D --> E[Attach repro log from section Terminal evidence]
Contributor checklist (click to expand)
  • I reproduced loss of workflow context after process exit
  • I confirmed AgentFlow runs without WorkflowStorageAdapter
  • I read Expected vs actual below
  • I reviewed Suggested code changes and linked modules

Architecture — current (in-memory)

flowchart LR
  subgraph Process
    AF[AgentFlow]
    WC[WorkflowContext in RAM]
    ST[Steps / patterns]
    AF --> ST --> WC
  end
  APP[approval module] -. agent-level only .-> AF
  TOOL[FunctionTool resumed flag] -. tool-level only .-> ST
  WC -->|process exit| LOST[(State lost)]
Layer Module Responsibility Survives restart?
Orchestration src/praisonai-ts/src/workflows/index.ts Step execution No
Context WorkflowContext Key/value per run No
Human gate src/praisonai-ts/src/approval/ Approve/deny actions Partial
Tool idempotency src/praisonai-ts/src/tools/decorator.ts resumed + isRestartSafe Per tool call only
Persistence src/praisonai-ts/src/db/ SQLite sessions Not wired to workflows

Architecture — proposed (durable)

sequenceDiagram
  participant Dev as Application
  participant Flow as AgentFlow
  participant Store as WorkflowStorageAdapter
  participant Op as Operator CLI

  Dev->>Flow: run(input, { storage })
  Flow->>Store: createRun + checkpoints each step
  Flow->>Flow: step requires approval
  Flow->>Store: saveCheckpoint(status=suspended)
  Note over Flow: process may stop here
  Op->>Flow: resume(runId, { approved: true })
  Flow->>Store: loadCheckpoint
  Flow->>Flow: continue from step_index
  Flow->>Store: mark completed
Component New / changed Storage
WorkflowStorageAdapter New interface Pluggable
SqliteWorkflowStorage New Reuse db() patterns
AgentFlow.run / resume Extend Optional adapter
AgentOS routes New HTTP resume

Expected vs actual behaviour

# Scenario Expected (product) Actual (today) Severity
1 Workflow suspends for approval Run id stable; status suspended in store Context only in memory High
2 Server restart during suspend Operator resumes with same run id Run lost; must restart workflow High
3 Parallel branch waits on human Only blocked branch suspended Entire flow blocked or undefined Medium
4 YAML workflow without storage Same as today (no regression) Works in-memory OK
5 CLI lists suspended runs JSON table of run ids No command Medium
6 Audit who resumed resumed_by persisted Not available Medium

Terminal evidence (reproduction)

Evidence A — workflow unit tests pass without persistence (expected today)
> jest tests/unit/workflows

PASS tests/unit/workflows/patterns.test.ts
PASS tests/unit/workflows/workflow.test.ts
PASS tests/unit/workflows/yaml-parser.test.ts

Test Suites: 13 passed, 13 total
Tests:       230 passed, 230 total

Interpretation: orchestration logic is sound; gap is durability, not step semantics.

Evidence B — conceptual repro script (suspend gap)

Run after building the SDK:

> node -e "
  const { AgentFlow } = require('./dist/workflows');
  const flow = new AgentFlow('approval-demo');
  flow.addStep({ name: 'need-approval', execute: async (ctx) => {
    ctx.set('pending', true);

> **Note:** Full diagrams, code patches, and terminal evidence live in `rich_sections.py` for this issue; this draft stays ~300 lines for review.

---

## Regression guardrails

| Check | Command |
|-------|---------|
| Unit tests | `npm test -- tests/unit/workflows` |
| Build | `npm run build` |

**Related:** `PAI-TS-001` cross-links other PAI-TS platform drafts in the same milestone.

## Problem statement

Long-running workflows fail or block when:
- A step requires human approval and the server process restarts.
- An external webhook will arrive asynchronously.
- Operators need to inspect intermediate state before continuing.

Today, in-memory workflow context is lost on crash. Partial mitigations (tool `resumed` flags, approval helpers) do not replace a persisted run record.

## Goals

1. Introduce `WorkflowRun` identity durable across restarts.
2. Allow any step to call `suspend(reason, payload)` and persist snapshot.
3. Allow `resume(runId, input)` from CLI, HTTP API, and programmatic SDK.
4. Integrate with existing SQLite adapter first; define port for Postgres.
5. Maintain backward compatibility: workflows without storage behave as today.

## Non-goals

- Distributed workflow engine across multiple regions in v1.
- Visual workflow editor (tracked separately).
- Replacing agent chat memory systems.
- Guaranteeing exactly-once side effects for non-idempotent tools without user configuration.

## Personas

| Persona | Need |
|---------|------|
| Platform engineer | Operate workflows in production with restart safety |
| Application developer | Simple API to pause for approval |
| Compliance reviewer | Audit who resumed and with what input |
| SRE | Metrics on suspended run age and resume latency |

## Persistence schema (logical)

| Column / field | Type | Description |
|----------------|------|-------------|
| run_id | UUID | Primary key |
| workflow_name | string | Registered flow name |
| status | enum | running/suspended/completed/failed |
| step_index | int | Current step pointer |
| context_json | json | Serializable workflow context |
| suspend_reason | string | Optional |
| suspend_payload | json | Operator-visible data |
| created_at | timestamp | UTC |
| updated_at | timestamp | UTC |
| resumed_by | string | Actor id from auth context |

## Security requirements

- SR-001: All resume operations require authenticated actor id in audit log.
- SR-002: Checkpoint payloads must not store raw secrets; use references only.
- SR-003: Cross-tenant run id access returns 404 without leaking existence.
- SR-004: Rate-limit resume API to prevent brute-force run id guessing.
- SR-005: Optional encryption at rest for context_json when enterprise flag set.

## Functional requirements

- FR-001: Persist checkpoint within 500ms of suspend call (p95).
- FR-002: Expose runId to application after first suspend.
- FR-003: Resume accepts optional operator input merged into context.
- FR-004: Failed step may suspend or fail based on workflow policy.
- FR-005: listSuspended returns paginated results sorted by suspendedAt desc.
- FR-006: Storage adapter interface documented and mockable in tests.

## Non-functional requirements

- NFR-001: Resume latency p95 under 2s for checkpoints under 1MB.
- NFR-002: SQLite adapter supports 10k suspended runs without full table scan on list.
- NFR-003: Checkpoint write must not block event loop >100ms without async offload.

## CLI requirements

- CLI-001: `praisonai-ts workflow list --suspended` returns JSON table.
- CLI-002: `praisonai-ts workflow resume <runId> --input file.json`.
- CLI-003: Exit code 2 when run not found or not suspended.

## HTTP API requirements (AgentOS)

- API-001: POST /workflow-runs/{id}/resume accepts JSON body.
- API-002: GET /workflow-runs/{id} returns status and suspend metadata.
- API-003: GET /workflow-runs?suspended=true supports cursor pagination.

## Integration test matrix

| ID | Scenario | Preconditions | Steps | Expected |
|----|----------|---------------|-------|----------|
| WF-SUSP-001 | Suspend mid-step then resume after simulated process restart | SDK built; storage adapter configured | Execute steps in test spec 1 | State consistent; typed errors; no data loss |
| WF-SUSP-002 | Reject resume when run id unknown | SDK built; storage adapter configured | Execute steps in test spec 2 | State consistent; typed errors; no data loss |
| WF-SUSP-003 | Parallel branch A suspended while branch B completes | SDK built; storage adapter configured | Execute steps in test spec 3 | State consistent; typed errors; no data loss |
| WF-SUSP-004 | Include pattern nested inside loop preserves step pointer | SDK built; storage adapter configured | Execute steps in test spec 4 | State consistent; typed errors; no data loss |
| WF-SUSP-005 | Checkpoint size near 1MB still loads under NFR budget | SDK built; storage adapter configured | Execute steps in test spec 5 | State consistent; typed errors; no data loss |
| WF-SUSP-006 | Concurrent resume requests are idempotent | SDK built; storage adapter configured | Execute steps in test spec 6 | State consistent; typed errors; no data loss |
| WF-SUSP-007 | Cancel suspended run transitions to cancelled status | SDK built; storage adapter configured | Execute steps in test spec 7 | State consistent; typed errors; no data loss |
| WF-SUSP-008 | YAML workflow without storage adapter unchanged behaviour | SDK built; storage adapter configured | Execute steps in test spec 8 | State consistent; typed errors; no data loss |

## Unit test matrix

| ID | Scenario | Preconditions | Steps | Expected |
|----|----------|---------------|-------|----------|
| WF-UNIT-001 | WorkflowContext JSON round-trip preserves keys | SDK built; storage adapter configured | Execute steps in test spec 1 | State consistent; typed errors; no data loss |
| WF-UNIT-002 | Storage adapter createRun + saveCheckpoint + loadCheckpoint | SDK built; storage adapter configured | Execute steps in test spec 2 | State consistent; typed errors; no data loss |
| WF-UNIT-003 | suspend() writes status suspended and reason | SDK built; storage adapter configured | Execute steps in test spec 3 | State consistent; typed errors; no data loss |
| WF-UNIT-004 | resume() rejects completed runs | SDK built; storage adapter configured | Execute steps in test spec 4 | State consistent; typed errors; no data loss |
| WF-UNIT-005 | listSuspended filters by workflow name | SDK built; storage adapter configured | Execute steps in test spec 5 | State consistent; typed errors; no data loss |
| WF-UNIT-006 | Serialization strips non-JSON values with clear error | SDK built; storage adapter configured | Execute steps in test spec 6 | State consistent; typed errors; no data loss |

## Migration notes

- MIG-001: Existing YAML workflows require no changes when storage unset.
- MIG-002: Opt-in: pass storage adapter in AgentFlow.run options only.
- MIG-003: v1 SQLite schema versioned; migration script for future columns.

## Documentation outline

- DOC-001: Sequence diagram for suspend/resume (Mintlify).
- DOC-002: Operator guide: list and resume suspended runs.
- DOC-003: Adapter implementation guide for Postgres.
- DOC-004: Security: authn/z on resume endpoints.
- DOC-005: Troubleshooting: stale locks and corrupted checkpoints.

## Proposed public API (summary)

`WorkflowStorageAdapter` with createRun, saveCheckpoint, loadCheckpoint, listSuspended.
`AgentFlow.run(input, { storage })` and `AgentFlow.resume(runId, input)`.
See **Architecture — proposed** in the triage section for sequence diagram.

## Acceptance criteria

- [ ] Suspend survives process kill and resume completes successfully.
- [ ] Parallel branch suspend resumes only affected branch.
- [ ] Unit tests cover serialization round-trip for WorkflowContext.
- [ ] Documentation published under docs/workflows/durable-execution.md.
- [ ] No breaking change for flows not passing storage adapter.

## Rollout plan

Phase 1: SQLite adapter + in-process API.
Phase 2: AgentOS routes + CLI.
Phase 3: Postgres adapter + migration tool.
Phase 4: Metrics and alerting hooks.

## Open questions

- OQ-001: Should nested Include patterns inherit the parent run id or allocate a child run id?
- OQ-002: How should parallel branches checkpoint: one shared snapshot or one per branch?
- OQ-003: What is the retention policy for suspended runs (TTL, manual purge, compliance hold)?
- OQ-004: Should resume require the same authenticated actor who created the run?
- OQ-005: How do we version checkpoint JSON when WorkflowContext schema evolves?
Lenguaje dominante
Python
Estrellas
9.3k
Forks
1.5k
Merge medio
1 d 5 h
PR fusionados (30 d)
520

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de MervinPraison/PraisonAI

Todos los issues de MervinPraison/PraisonAI

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.