feat(testing): emulator-backed integration tier (ADR-013 Tier 2a), with fixes for the defects found
维护者通常 1 天内回复
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 15/100
调研方向
The issue describes a tier spanning CDK, CLI and agent suites, a new emulator workflow, documentation, and numerous behavioral fixes; the linked pull request #1048 is already open. Start with ADR-013 and the issue’s acceptance criteria, then review the pull request and its emulator research report at docs/research/aws-emulator-local-tests.md. Done means the listed integration, guard, workflow and build criteria pass, with unit coverage for behavior changing fixes.
由索引模型根据 Issue 内容生成。
描述
Component
Other: test tooling and CI. The fixes that come with it touch CDK handlers and constructs, the agent runtime and the CLI.
Describe the feature
An integration test tier that runs ABCA's real handler, CLI and agent code against the AWS Emulator. The code runs in jest and pytest as usual; only its AWS SDK calls leave the process, to a pinned emulator on loopback. It needs Docker and no AWS account.
This is Tier 2a of ADR-013 (container integration tests, between the local unit build and remote CI), which the ADR marks "does not exist". It starts as an advisory check: a workflow runs it on every pull request and in the merge queue, and it is not a required status check. It does not replace the real-AWS integ smoke.
Use case
Between the mocked unit suites and the real-AWS integ smoke, nothing runs ABCA's code against an AWS API. The unit suites mock the SDK at module level, so they cannot see what DynamoDB does with a condition, an update expression, an index key or Limit together with a filter. The integ smoke sits behind an approval gate and evaluates one conditional write.
An investigation of the emulator (the report is docs/research/aws-emulator-local-tests.md in the pull request) built a proof of concept of the tier. It reproduced #51 and #1003 and found further defects on main that no issue tracked.
Proposed solution
One pull request, from one branch:
mise run test:integrationstarts a pinned, signature-verified emulator in a container, runs thecdk,cliand agent emulator suites, and stops it. The suites refuse to run against anything but the emulator, and the unit suites cannot pick them up.- Tests of those refusals in the unit suites, so that a guard that stops refusing fails the required build.
- An
emulatorworkflow that runs the tier on every pull request and in the merge queue, not required. - The rules for tests in the tier and its known differences from AWS, documented beside the tooling; ADR-013 and the
AGENTS.mdguides point to it. - Fixes for defects the investigation found. Not fixed here: those an open pull request addresses, and a few that stay open; the Outcome section of the report lists both. Each fix that changes behaviour has a unit test that fails without it; the comment, type and documentation corrections have none.
- #51: the task list is not newest first, and a filtered page can come back empty while matching tasks exist. The fix adds an index,
UserCreatedAtIndex, to the task table. On an existing stack the handler reads the old index, in its old order, until DynamoDB reports the new oneACTIVE; the deployment guide gets an entry. Known limit: a filtered list reads at most ten queries per request (about 210 rows at the default page size) andbgagent listdoes not follownext_token, so a match further back still shows as an empty list with "(More results available)". GET /v1/webhooksapplies its filter afterLimit, as #51 did.- The stranded-approval sweep measures from the start of
RUNNING, because the agent's approval transitions do not maintainstatus_created_at. - A session role that cannot be assumed does not stop the task.
- The trajectory writer never retries the creation of its log stream.
- Four writers of a terminal status leave out an attribute a terminal row needs: the Slack cancel button, the presign-failure path of
create-task, the stranded-task sweep (nottl) and confirm-uploads when screening blocks an attachment (nocompleted_at). - The link handlers for Slack, Linear and Jira check a code's
ttlthemselves. - The confirm-uploads Lambda never receives the per-user concurrency cap.
- CLI: a GitHub token secret that was never configured is reported as configured;
ops stuck-tasksages tasks by creation time;ops concurrencyshows a default limit of 3 where the stack deploys 10. - The
integjob's command has been unrunnable since #567: it passescdk deployarguments tointeg-runner. - Comments and types that blame the SDK for numbers the agent stores as strings, and an order dependence in the agent unit suite.
- Documentation that gives a default limit of 3, or describes behaviour from before #441 (the event
admission_rejected, a task failed at the concurrency limit), and comments that give the Linear webhook dedup window as 60 seconds.
- #51: the task list is not newest first, and a filtered page can come back empty while matching tasks exist. The fix adds an index,
Other information
Acceptance criteria:
-
mise run test:integrationpasses with no AWS account or credential, andmise run builddoes not run it. - The unit suites test the guard's refusals.
- The
emulatorworkflow runs on pull requests and in the merge queue and is not a required check. - Each fix that changes behaviour has a unit test that fails without the fix.
-
mise run buildpasses.
Behaviour that changes besides the fixes themselves. None of it changes the shape of an API response:
ListTasksFngets 15 s and 256 MB (it had Lambda's 3 s and 128 MB). It asksDescribeTablefor the status of the new index on each request that starts a list until the answer isACTIVE, then not again in that execution environment; a failed check is a500.age_secondsinbgagent ops stuck-tasks --output jsonbecomes the time in the current status, not the time since creation.bgagent github set-webhook-secretno longer reads the secret first and always prints its notice.- An STS failure at task start that outlasts the SDK's retries fails the task; before, the task ran on and lost one best-effort write.
bgagent platform doctorfails the token check for a stored value that is not in a GitHub token format.- A link code past its
ttlis refused from the deploy on.
Not verified: the emulator workflow has not run on GitHub, and nothing was compared with real AWS, including the deploy that adds the index.
Not in scope, left to open pull requests: #904 (heartbeat during approval waits), #1009 (seat release and the statuses the reconciler counts; #1003), #1013 (its new approval writer needs status_created_at too).
Related: the pull request fixes #51. Issue #61 asks for a trace round-trip test on moto; the tier has that test on the emulator, and its README covers what #61 asks of a README, at another path.
Acknowledgements
- I may be able to implement this feature
- This might be a breaking change
- 主要语言
- TypeScript
- 星标
- 173
- 派生
- 53
- 平均合并
- 2 天 9 小时
- 30 天内合并 PR
- 14
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
aws-samples/sample-autonomous-cloud-coding-agents 的其他 Issue
-
bug infra-cdk
难度 2/5 1-3 小时 新手友好度 78/100
aws-samples/sample-autonomous-cloud-coding-agents#1044 ·
维护者通常 1 天内回复
-
cdk: support Docker-compatible engines such as Finch (clean:disk hard-codes docker)可能已有人在做 @alvarohare 于 20 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
aws-samples/sample-autonomous-cloud-coding-agents#969 · 1 条评论 · 1 个 reaction · 已指派 1 人 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
aws-samples/sample-autonomous-cloud-coding-agents#968 ·
维护者通常 1 天内回复
-
documentation P2 security
难度 2/5 1-2 天 新手友好度 74/100
aws-samples/sample-autonomous-cloud-coding-agents#793 ·
维护者通常 1 天内回复
-
Docs: How to clear previously set attributes for a blueprint可能重新可做 @rkumarus-aws 于 77 天前认领,目前没有进行中的 PR。 未关闭documentation
难度 2/5 1-3 小时 新手友好度 68/100
aws-samples/sample-autonomous-cloud-coding-agents#767 · 2 条评论 ·
维护者通常 1 天内回复
查看 aws-samples/sample-autonomous-cloud-coding-agents 的全部 Issue
相似的 Issue
-
area: batch
难度 2/5 1-3 小时 新手友好度 75/100
inabagumi/shinju-date#7294 ·
维护者通常 1 天内回复
-
documentation
难度 2/5 1-3 小时 新手友好度 68/100
streaming-video-technology-alliance/common-media-library#700 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
vercel-labs/ui#809 ·
维护者通常 1 天内回复
-
enhancement
难度 2/5 1-3 小时 新手友好度 67/100
KrisCris/village-in-the-shade-guide#75 ·
维护者通常 1 天内回复
-
bot:ai-assisted component:platform-js status:untriaged
难度 1/5 1 小时以内 新手友好度 74/100
midnightntwrk/platform-js#38 ·