[Bug]: Virtual child contexts exit replay early and omit resumed operation notifications
Les mainteneurs répondent en général sous 1 jour
Personne n'a encore pris cette issue.
- #909 par @nandanadileep — fermée sans fusion
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 55/100
- Type d'issue
- Bug
- Clarté
- Clairement spécifiée
- Activité
- Active
- Stack technique
- python
- Domaine
- distributed-systems
Piste de recherche
Start with _replay_aware and run_in_child_context in packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/context.py, then trace how create_child_context inherits replay status and how the operation-update gate uses it. Run the focused reproduction in the issue and compare virtual and ordinary children. Done means resumed virtual children deliver the updated wait notification once, while genuinely new work transitions to NEW and checkpoint semantics remain unchanged.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Expected Behavior
Entering run_in_child_context(..., config=ChildConfig(is_virtual=True)) during replay must not treat the absence of the virtual container's checkpoint as evidence of new work. A virtual container intentionally has no checkpoint, while its inner operations can have persisted history.
When the next inner operation is an existing successful wait reported as updated since the previous invocation, the child should retain replay status long enough to consume that history and deliver the normal operation-update notification. It can then transition to NEW at the actual replay boundary.
Actual Behavior
The core SDK changes the parent from REPLAY to NEW before creating the virtual child. The child inherits NEW, so its existing wait bypasses the replay/update notification gate.
A core-only comparison, with the same completed wait in the supplied state, produces:
{'virtual': False, 'child_enters_replay': True, 'wait_update_hooks': ['virtual-wait'], 'result': 'ok'}
{'virtual': True, 'child_enters_replay': False, 'wait_update_hooks': [], 'result': 'ok'}
Both calls return the stored successful wait outcome. The demonstrated defect is replay-state classification and the omitted notification; this reproduction does not demonstrate a failed wait or repeated business side effects. An observable consequence is missing terminal wait telemetry in Execution view when a virtual child suspends and resumes. Other consumers of replay status, including replay-aware logging, may also be affected, but that broader impact has not been validated here.
Steps to Reproduce
The motivating workflow is a virtual child running step -> durable wait (5 s) -> step, with the wait completing between invocations. On resume, the virtual container has no checkpoint by design, but its inner operations do.
The following focused reproduction isolates the core boundary without AWS credentials, an OTel plugin, or an exporter. It uses the real DurableContext.run_in_child_context and wait implementations, with a mocked execution-state boundary that supplies a successful updated wait. The ordinary-child control has a STARTED container checkpoint; the virtual child correctly has none.
Run from the Python SDK repository root at the tested revision:
PYTHONDONTWRITEBYTECODE=1 hatch run dev-otel:python - <<'PYTHON'
from datetime import UTC, datetime
from unittest.mock import Mock
from aws_durable_execution_sdk_python.context import DurableContext, ExecutionContext
from aws_durable_execution_sdk_python.config import Duration, ChildConfig
from aws_durable_execution_sdk_python.state import ReplayStatus, CheckpointedResult
from aws_durable_execution_sdk_python.lambda_service import (
Operation, OperationType, OperationStatus, OperationSubType,
)
for virtual in (False, True):
state = Mock()
state.wrap_user_function.side_effect = lambda func, *args, **kwargs: func
state.durable_execution_arn = 'test-arn'
parent = DurableContext(
state, ExecutionContext('test-arn'), replay_status=ReplayStatus.REPLAY
)
context_id = parent._peek_next_operation_id()
child_probe = parent.create_child_context(context_id, is_virtual=virtual)
wait_id = child_probe._peek_next_operation_id()
timestamp = datetime(2026, 10, 30, tzinfo=UTC)
operations = {
wait_id: Operation(
operation_id=wait_id, operation_type=OperationType.WAIT,
status=OperationStatus.SUCCEEDED, name='virtual-wait',
parent_id=None if virtual else context_id,
sub_type=OperationSubType.WAIT,
start_timestamp=timestamp, end_timestamp=timestamp,
)
}
if not virtual:
operations[context_id] = Operation(
operation_id=context_id, operation_type=OperationType.CONTEXT,
status=OperationStatus.STARTED, name='virtual-context',
parent_id=None, sub_type=OperationSubType.RUN_IN_CHILD_CONTEXT,
start_timestamp=timestamp,
)
state.get_checkpoint_result.side_effect = lambda op_id: (
CheckpointedResult.create_from_operation(operations[op_id])
if op_id in operations else CheckpointedResult()
)
state.is_operation_updated_since_last_invocation.side_effect = (
lambda op_id: op_id == wait_id
)
child_states = []
def body(child):
child_states.append(child.is_replaying())
child.wait(Duration.from_seconds(5), name='virtual-wait')
return 'ok'
result = parent.run_in_child_context(
body, name='virtual-context', config=ChildConfig(is_virtual=virtual)
)
emitted_wait = [
call.args[0].name
for call in state.emit_operation_update_hook.call_args_list
]
print({
'virtual': virtual,
'child_enters_replay': child_states[0],
'wait_update_hooks': emitted_wait,
'result': result,
})
PYTHON
dev-otel is only the available development environment used to run this probe; the script imports core SDK modules and the Python standard library only. This is an isolated core reproduction, not a new cloud or end-to-end OTel validation.
SDK Version
Runtime reproduction: source checkout 44b9f4d1c862b50041423e19fe6a250bbf7bd72f, whose core package declares 3.0.0.
The same checkpoint-existence predicate and child replay-state inheritance remain in current main 4625d01bbfa97da67050014851af7eb41c61a842, as checked by source comparison on 2026-10-30. The runtime probe above was executed at the first revision; it does not claim a full runtime validation of the later revision.
Python Version
3.15.2, using the existing Hatch development environment on macOS. Other Python versions were not rerun for this report.
Is this a regression?
Unknown; no last-known-working version has been established.
Additional Context
The source path at the reproduced revision is:
_replay_aware, context.py:600: while replaying,not next_existsimmediately calls_set_replay_status_new(), including for a virtual container.run_in_child_context, context.py:833: the child is created inside that replay-aware scope, after the transition.create_child_context, context.py:444: the child inherits its creator's replay status.- The operation-update/replay gate, context.py:564, runs only while replaying. The inner wait bypasses it and returns its stored success.
Suggested acceptance criteria:
- Preserve replay state through virtual-container entry when its inner history still needs replay; switch to NEW when actual new work is reached.
- Exercise initial invocation and resume with
step -> wait -> step, checking that completed step bodies do not rerun and the updated wait's normal core notification is delivered once. - Cover nested virtual contexts, ordinary-child controls, and a virtual context whose next inner operation is genuinely new.
- Preserve virtual checkpoint semantics, stable operation IDs, and wire
ParentId; do not fabricate a container checkpoint to obtain replay state. - Keep rejecting a non-virtual container checkpoint encountered under a virtual child, as required by #704.
Related but separate:
- #854 / #869 address discovery and export of visible terminal completions in OTel. Plugin-side recovery can mitigate missing telemetry but does not correct this core replay-state transition.
- #704 concerns replay compatibility when switching a previously checkpointed non-virtual child to virtual.
- #376 concerns invocation-initialization replay state when history is paginated.
This issue is scoped to the Python core replay boundary. Virtual-span ancestry, cross-invocation start-time reconstruction, and execution-wide virtual-span export deduplication should be tracked separately.
- Langage dominant
- Python
- Étoiles
- 59
- Forks
- 32
- Merge moyen
- 1 j 5 h
- PR mergées (30 j)
- 76
Préparer son environnement
- Aucun Dockerfile ni fichier Docker Compose
- Propose un modèle de pull request
- Lire le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de aws/aws-durable-execution-sdk-python
-
needs-triage
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
aws/aws-durable-execution-sdk-python#977 ·
Les mainteneurs répondent en général sous 1 jour
-
[AgentCore session] Remove the unused session timeoutPeut-être pris @srivilliamsai l’a pris aujourd’hui. Ouverteneeds-triage
Difficulté 2/5 1-3 heures Accessibilité débutants 77/100
aws/aws-durable-execution-sdk-python#970 · 1 commentaire ·
Les mainteneurs répondent en général sous 1 jour
-
enhancement needs-triage
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
aws/aws-durable-execution-sdk-python#952 ·
Les mainteneurs répondent en général sous 1 jour
-
[AgentCore session] W4: Document sessions in the worker READMEPeut-être pris @ParidelPooya l’a pris il y a 3 jours. Ouverteenhancement needs-triage
Difficulté 1/5 1-3 heures Accessibilité débutants 76/100
aws/aws-durable-execution-sdk-python#950 ·
Les mainteneurs répondent en général sous 1 jour
-
[AgentCore session] W3: Add the session ID and job count to worker logsPeut-être pris @ParidelPooya l’a pris il y a 3 jours. Ouverteenhancement needs-triage
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
aws/aws-durable-execution-sdk-python#949 ·
Les mainteneurs répondent en général sous 1 jour
Toutes les issues de aws/aws-durable-execution-sdk-python
Issues similaires
-
Difficulté 2/5 1-3 heures Accessibilité débutants 73/100
Les mainteneurs répondent en général sous 1 jour
-
Unused loot bag group names left in dreamcraft en_US.langPeut-être pris Une pull request liée à cette issue est ouverte ou déjà fusionnée. OuverteStatus: Triage Suggestion
Difficulté 1/5 Moins d'une heure Accessibilité débutants 90/100
GTNewHorizons/GT-New-Horizons-Modpack#27789 ·
Les mainteneurs répondent en général sous 1 jour
-
bug frontend sev-5
Difficulté 2/5 1-3 heures Accessibilité débutants 83/100
-
core good first issue
Difficulté 2/5 1-3 heures Accessibilité débutants 65/100
-
b4b code health next
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
ESCOMP/CTSM#4330 · 1 commentaire ·
Les mainteneurs répondent en général sous 1 jour