Plugin conformance tests are flaky when CloudWatch logs are not yet visible
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- aws, python
- Domain
- ci-cd, observability, testing
Research direction
Start with .github/workflows/conformance-tests.yml and the “Run conformance suite” step, then rerun the plugin suite repeatedly against the same commit. Investigate CloudWatch propagation or log-stream lookup timing and capture the suggested log-group, execution-filter, event-count, and retrieved-record diagnostics; done means the conformance tests reliably find expected records.
Written by the indexing model from the issue text.
Description
Summary
The conformance (plugin) job can fail because expected plugin log records are not returned by CloudWatch, even though the durable execution and history validation complete successfully.
Evidence
- Previous successful run:
33907972521at commit4ce54bcf. - Failed run:
33922277509at commit676c75a. - Attempt 1, job
101185043031: test10-11(plugin_nested_parent_linkage) failed because both expectedoperation-endrecords matched 0 logs: the inner step record and the child-context record. - Retry attempt 2, job
101188790974: test10-11passed, but unrelated test10-7(plugin_terminal_failure) failed because itsinvocation-startandinvocation-endrecords matched 0 logs.
The failure moving between tests on the same commit strongly suggests CloudWatch ingestion/filter timing or log-stream lookup flakiness rather than a deterministic SDK regression. The diff from the previous successful commit only changes OTel release metadata and does not affect the regular core plugin suite.
Suggested investigation
- Increase or make configurable the CloudWatch log propagation/polling window in the conformance runner.
- Log the resolved log group, execution ARN filter, event count, and final retrieved records when
ExpectedLogsfails. - Pin the conformance runner to a known commit in CI instead of installing
@main. - Re-run the plugin suite repeatedly against the same commit to confirm the failure rate.
Relevant workflow: .github/workflows/conformance-tests.yml, step Run conformance suite.
- Dominant language
- Python
- Stars
- 59
- Forks
- 32
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 76
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aws/aws-durable-execution-sdk-python
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
aws/aws-durable-execution-sdk-python#977 ·
Maintainers usually reply within 1 day
-
[AgentCore session] Remove the unused session timeoutPossibly taken @srivilliamsai claimed this today. Openneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
aws/aws-durable-execution-sdk-python#970 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws/aws-durable-execution-sdk-python#952 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W4: Document sessions in the worker READMEPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 1/5 1-3 hours Newbie friendliness 76/100
aws/aws-durable-execution-sdk-python#950 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W3: Add the session ID and job count to worker logsPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
aws/aws-durable-execution-sdk-python#949 ·
Maintainers usually reply within 1 day
All issues in aws/aws-durable-execution-sdk-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Unused loot bag group names left in dreamcraft en_US.langPossibly taken A pull request linked to this issue is open or already merged. OpenStatus: Triage Suggestion
Difficulty 1/5 Under an hour Newbie friendliness 90/100
GTNewHorizons/GT-New-Horizons-Modpack#27789 ·
Maintainers usually reply within 1 day
-
bug frontend sev-5
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
-
core good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
b4b code health next
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ESCOMP/CTSM#4330 · 1 comment ·
Maintainers usually reply within 1 day