[Bug]: Emulator retries invalid invocation outputs that the service fails immediately

未关闭
#670 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
72/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python
领域
backend

调研方向

从 aws_durable_execution_sdk_python_testing.executor.Executor._finish_invocation() 开始,跟踪 _validate_invocation_response_and_store() 和 _retry_invocation()。复现列出的无效输出情况和 PENDING 情况,然后添加回归覆盖,以展示 fail-fast 行为、InvalidParameterValueException 错误结构、PENDING 的三次尝试上限,以及从一次暂时性无效输出中恢复。

由索引模型根据 Issue 内容生成。

描述

bug parity pkg:testing
Expected Behavior

The emulator should mirror the service's handling of invalid invocation outputs. Verified service behavior:

Most invalid outputs fail the execution immediately, with no retry. This applies to:

  • Status=FAILED with a Result present
  • Status=SUCCEEDED with an Error present
  • a missing or unrecognized Status
  • an unparseable/malformed invocation output payload

In each case the service fails the execution with an error object of the form:

{
  "ErrorType": "InvalidParameterValueException",
  "ErrorMessage": "<the specific validation message>"
}

There is exactly one retried case: Status=PENDING when the execution has no pending operations. The service treats this as a transient runtime-level error (it can arise from SDK race conditions that resolve on replay) and retries the invocation, up to 3 consecutive attempts, before failing the execution with the same InvalidParameterValueException-shaped error.

Actual Behavior

In aws_durable_execution_sdk_python_testing.executor.Executor._finish_invocation(), all validation failures from _validate_invocation_response_and_store() are routed through the same retry path:

except (InvalidParameterValueException, IllegalStateException) as e:
    ...
    self._set_invocation_gate(execution_arn, InvocationState.PRE_INVOKE)
    self._retry_invocation(execution, error_obj)
    return

_retry_invocation re-invokes the handler up to MAX_CONSECUTIVE_FAILED_ATTEMPTS = 5 times with a flat RETRY_BACKOFF_SECONDS = 5 backoff before failing.

Consequences:

  • A deterministically invalid output (e.g. SUCCEEDED with an Error) is re-invoked 5 times before failing, where the service fails on the first response. For a deterministic handler this adds ~25 seconds and 4 pointless invocations, and every re-invocation replays the handler.
  • The one case the service does retry (PENDING with no pending operations) gets 5 attempts instead of 3.
  • The terminal error surfaced to the customer is ErrorObject.from_exception(e) rather than the service's {ErrorType: "InvalidParameterValueException", ErrorMessage: ...} shape.
Proposed Fix

In _finish_invocation, distinguish the two classes:

  • Fail-fast validation errors (FAILED+Result, SUCCEEDED+Error, missing/unknown status, malformed output): fail the execution immediately with an InvalidParameterValueException-typed error object carrying the validation message.
  • PENDING with no pending operations: keep the retry path, with a consecutive-attempt cap of 3.

Regression tests:

  • each fail-fast case produces a FAILED execution after a single invocation, with the InvalidParameterValueException error shape;
  • PENDING with no pending operations retries and fails after 3 consecutive attempts;
  • a transient case (invalid output once, then a valid PENDING/SUCCEEDED) recovers without failing the execution.
Related
SDK Version

aws-durable-execution-sdk-python-testing current main as of 2026-08-21.

Python Version

3.13

Is this a regression?

No known working version.

主要语言
Python
星标
59
派生
32
平均合并
1 天 5 小时
30 天内合并 PR
76

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

aws/aws-durable-execution-sdk-python 的其他 Issue

查看 aws/aws-durable-execution-sdk-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。