feat(testing): emulator-backed integration tier (ADR-013 Tier 2a), with fixes for the defects found
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 15/100
Línea de trabajo
The issue describes a tier spanning CDK, CLI and agent suites, a new emulator workflow, documentation, and numerous behavioral fixes; the linked pull request #1048 is already open. Start with ADR-013 and the issue’s acceptance criteria, then review the pull request and its emulator research report at docs/research/aws-emulator-local-tests.md. Done means the listed integration, guard, workflow and build criteria pass, with unit coverage for behavior changing fixes.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Component
Other: test tooling and CI. The fixes that come with it touch CDK handlers and constructs, the agent runtime and the CLI.
Describe the feature
An integration test tier that runs ABCA's real handler, CLI and agent code against the AWS Emulator. The code runs in jest and pytest as usual; only its AWS SDK calls leave the process, to a pinned emulator on loopback. It needs Docker and no AWS account.
This is Tier 2a of ADR-013 (container integration tests, between the local unit build and remote CI), which the ADR marks "does not exist". It starts as an advisory check: a workflow runs it on every pull request and in the merge queue, and it is not a required status check. It does not replace the real-AWS integ smoke.
Use case
Between the mocked unit suites and the real-AWS integ smoke, nothing runs ABCA's code against an AWS API. The unit suites mock the SDK at module level, so they cannot see what DynamoDB does with a condition, an update expression, an index key or Limit together with a filter. The integ smoke sits behind an approval gate and evaluates one conditional write.
An investigation of the emulator (the report is docs/research/aws-emulator-local-tests.md in the pull request) built a proof of concept of the tier. It reproduced #51 and #1003 and found further defects on main that no issue tracked.
Proposed solution
One pull request, from one branch:
mise run test:integrationstarts a pinned, signature-verified emulator in a container, runs thecdk,cliand agent emulator suites, and stops it. The suites refuse to run against anything but the emulator, and the unit suites cannot pick them up.- Tests of those refusals in the unit suites, so that a guard that stops refusing fails the required build.
- An
emulatorworkflow that runs the tier on every pull request and in the merge queue, not required. - The rules for tests in the tier and its known differences from AWS, documented beside the tooling; ADR-013 and the
AGENTS.mdguides point to it. - Fixes for defects the investigation found. Not fixed here: those an open pull request addresses, and a few that stay open; the Outcome section of the report lists both. Each fix that changes behaviour has a unit test that fails without it; the comment, type and documentation corrections have none.
- #51: the task list is not newest first, and a filtered page can come back empty while matching tasks exist. The fix adds an index,
UserCreatedAtIndex, to the task table. On an existing stack the handler reads the old index, in its old order, until DynamoDB reports the new oneACTIVE; the deployment guide gets an entry. Known limit: a filtered list reads at most ten queries per request (about 210 rows at the default page size) andbgagent listdoes not follownext_token, so a match further back still shows as an empty list with "(More results available)". GET /v1/webhooksapplies its filter afterLimit, as #51 did.- The stranded-approval sweep measures from the start of
RUNNING, because the agent's approval transitions do not maintainstatus_created_at. - A session role that cannot be assumed does not stop the task.
- The trajectory writer never retries the creation of its log stream.
- Four writers of a terminal status leave out an attribute a terminal row needs: the Slack cancel button, the presign-failure path of
create-task, the stranded-task sweep (nottl) and confirm-uploads when screening blocks an attachment (nocompleted_at). - The link handlers for Slack, Linear and Jira check a code's
ttlthemselves. - The confirm-uploads Lambda never receives the per-user concurrency cap.
- CLI: a GitHub token secret that was never configured is reported as configured;
ops stuck-tasksages tasks by creation time;ops concurrencyshows a default limit of 3 where the stack deploys 10. - The
integjob's command has been unrunnable since #567: it passescdk deployarguments tointeg-runner. - Comments and types that blame the SDK for numbers the agent stores as strings, and an order dependence in the agent unit suite.
- Documentation that gives a default limit of 3, or describes behaviour from before #441 (the event
admission_rejected, a task failed at the concurrency limit), and comments that give the Linear webhook dedup window as 60 seconds.
- #51: the task list is not newest first, and a filtered page can come back empty while matching tasks exist. The fix adds an index,
Other information
Acceptance criteria:
-
mise run test:integrationpasses with no AWS account or credential, andmise run builddoes not run it. - The unit suites test the guard's refusals.
- The
emulatorworkflow runs on pull requests and in the merge queue and is not a required check. - Each fix that changes behaviour has a unit test that fails without the fix.
-
mise run buildpasses.
Behaviour that changes besides the fixes themselves. None of it changes the shape of an API response:
ListTasksFngets 15 s and 256 MB (it had Lambda's 3 s and 128 MB). It asksDescribeTablefor the status of the new index on each request that starts a list until the answer isACTIVE, then not again in that execution environment; a failed check is a500.age_secondsinbgagent ops stuck-tasks --output jsonbecomes the time in the current status, not the time since creation.bgagent github set-webhook-secretno longer reads the secret first and always prints its notice.- An STS failure at task start that outlasts the SDK's retries fails the task; before, the task ran on and lost one best-effort write.
bgagent platform doctorfails the token check for a stored value that is not in a GitHub token format.- A link code past its
ttlis refused from the deploy on.
Not verified: the emulator workflow has not run on GitHub, and nothing was compared with real AWS, including the deploy that adds the index.
Not in scope, left to open pull requests: #904 (heartbeat during approval waits), #1009 (seat release and the statuses the reconciler counts; #1003), #1013 (its new approval writer needs status_created_at too).
Related: the pull request fixes #51. Issue #61 asks for a trace round-trip test on moto; the tier has that test on the emulator, and its README covers what #61 asks of a README, at another path.
Acknowledgements
- I may be able to implement this feature
- This might be a breaking change
- Lenguaje dominante
- TypeScript
- Estrellas
- 173
- Forks
- 53
- Merge medio
- 2 d 9 h
- PR fusionados (30 d)
- 14
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de aws-samples/sample-autonomous-cloud-coding-agents
-
bug infra-cdk
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
aws-samples/sample-autonomous-cloud-coding-agents#1044 ·
Los mantenedores suelen responder en 1 día
-
cdk: support Docker-compatible engines such as Finch (clean:disk hard-codes docker)Posiblemente ocupada @alvarohare la tomó hace 20 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
aws-samples/sample-autonomous-cloud-coding-agents#969 · 1 comentario · 1 reacción · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
aws-samples/sample-autonomous-cloud-coding-agents#968 ·
Los mantenedores suelen responder en 1 día
-
documentation P2 security
Dificultad 2/5 1-2 días Aptitud para principiantes 74/100
aws-samples/sample-autonomous-cloud-coding-agents#793 ·
Los mantenedores suelen responder en 1 día
-
Docs: How to clear previously set attributes for a blueprintQuizá libre de nuevo @rkumarus-aws la tomó hace 77 días y no hay ningún pull request abierto. Abiertodocumentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
aws-samples/sample-autonomous-cloud-coding-agents#767 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de aws-samples/sample-autonomous-cloud-coding-agents
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
vercel-labs/ui#809 ·
Los mantenedores suelen responder en 1 día
-
bot:ai-assisted component:platform-js status:untriaged
Dificultad 1/5 Menos de una hora Aptitud para principiantes 74/100
midnightntwrk/platform-js#38 ·
-
[bug] Second false positive on high-cost plugin alert: detection only compares patch entry id, community plugin dsh-auto | [bug] 高成本插件告警第二次误报:检测只比对 patch 条目 id,社区插件 dsh-auto-review 被认成 @deepseek-ai/dsh-experimental-auto-reviewPosiblemente ocupada Un pull request vinculado a esta issue está abierto o ya se fusionó. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
ranxianglei/billion-context#7068 ·
Los mantenedores suelen responder en 1 día
-
vendor: bump whisper.cpp past 6a8addd — the current pin aborts the app on a failed Metal initAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
mybigday/whisper.rn#348 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
six-group/six-webcomponents#600 ·