[Bug]: Invalid suspension logs a WARN and returns a PENDING result that the service rejects

Geschlossen
  • Stimmt etwas nicht?
#753 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
48/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
aws, java
Bereich
backend, testing-qa

Rechercherichtung

Start with ExecutionManager.preSuspendCheck(), including its callers in deregisterActiveThread and finishCheckpointProcessing, then inspect DurableExecutor's retry path and LocalDurableTestRunner.runUntilComplete. Reproduce case 1 from #752 and trace the PENDING output. Done means invalid suspension is reported with awaited operation details, transient retries remain possible, the no-token path is allowed, and local tests expose invalid output.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

bug pkg:sdk
Expected Behavior

When the SDK suspends with no pending operation, it returns an output that the service rejects. The SDK should report this state as an SDK error. The error should name the operations that the suspended threads were waiting for.

The invocation should stay retryable. The race in #370 reached this state, and the execution still succeeded on a later invocation.

Actual Behavior
  1. ExecutionManager calls preSuspendCheck() before it suspends. It does so in deregisterActiveThread and in finishCheckpointProcessing.
  2. preSuspendCheck() looks for a pending operation: a STEP in PENDING, a WAIT or CALLBACK in STARTED, or a CHAINED_INVOKE in PENDING or STARTED.
  3. If it finds none, it logs one WARN line: Invalid suspension. No operation is pending. The line does not name an operation or a thread.
  4. The SDK then suspends and returns Status: PENDING with no error. The Lambda invocation succeeds, because the handler returns normally.
  5. The service rejects the output with InvalidParameterValueException: Cannot return PENDING status with no pending operations.
  6. If the cause is deterministic, every replay reaches the same state. The execution fails, and GetDurableExecution reports Status: FAILED with that error.

The customer sees Lambda invocations that succeed, one WARN line per invocation, and a failed execution. None of these names the operation or the SDK defect that caused the failure.

Steps to Reproduce

  1. Run case 1 from #752 with LocalDurableTestRunner. That handler runs a firstSuccessful() parallel with maxConcurrency(1), then calls get() on the skipped second branch.

  2. Each invocation returns status PENDING with no error. No operation is pending.

  3. Each invocation logs:

    WARN software.amazon.lambda.durable.execution.ExecutionManager - Invalid suspension. No operation is pending

  4. LocalDurableTestRunner.runUntilComplete returns PENDING. It does not report that the service would reject the output.

SDK Version

2.2.1. Reproduced on main at ef88276 (2.2.2-SNAPSHOT).

Java Version

21

Is this a regression?

Unknown

Additional Context

Three SDK defects have reached this state. Each ended with the service error above.

  • #370: a race between checkpoint completion and a waiting thread. Fixed in #659.
  • #736: a race in allCompleted() branch registration. Fixed in #737.
  • #752: get() on a SKIPPED parallel branch. Open.

A fix needs to handle three things.

  1. Keep the retry. A transient race can clear on a later invocation, as in #370. DurableExecutor already rethrows a retryable UnrecoverableDurableExecutionException. Its comment says this is to "let the backend retry the invocation". An error on that path could keep the retry and name the cause.
  2. Allow the no-token path. #706 proposes a PENDING return when a checkpoint response has no token. That path abandons in-flight operations, so the state can hold no pending operation. A stricter check must not reject that return.
  3. Report it in local tests. LocalDurableTestRunner returns PENDING for this output. It could report the output as invalid, so that a local test shows the outcome the service produces.

A smaller change keeps the current behavior and logs at ERROR with the ID and name of the awaited operation.

Vorherrschende Sprache
Java
Sterne
28
Forks
16
Ø Merge
2 T. 59 Min.
Gemergte PRs (30 T.)
62

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus aws/aws-durable-execution-sdk-java

Alle Issues in aws/aws-durable-execution-sdk-java

Ähnliche Issues

Weitere Issues zu Java

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.