[Feature]: Harden FileSystemSerDes publication and reference validation
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- backend, distributed-systems
Research direction
Start at FileSystemSerDes and the configured inner value SerDes, then compare the Java SDK filesystem SerDes work in aws/aws-durable-execution-sdk-java#648. Review the cloud coverage tracked by #527 for immutable publication and ownership/integrity scenarios. Done means versioned owner-bound envelopes, safe immutable and integrity-checked reads, legacy compatibility, preserved modes, exception reserialization, and lifecycle documentation.
Written by the indexing model from the issue text.
Description
What would you like?
Harden FileSystemSerDes so checkpoint file references remain immutable, integrity-checked, and bound to the durable payload that produced them.
The current implementation writes to a deterministic <operation_id>.json path using "w" and stores an unversioned {"file": ...} envelope. A later serialization for the same operation can overwrite content referenced by an older checkpoint, and deserialization trusts the file path carried in the envelope.
The desired behavior is:
- publish each file payload immutably so an existing checkpoint never observes replaced content;
- use a versioned, self-identifying filesystem envelope;
- bind the envelope to its durable execution and entity identity;
- verify file integrity and path safety before reading;
- retain
ALWAYS,OVERFLOW, preview, URI, and hash modes.
Possible Implementation
- Serialize the configured value SerDes once, compute a SHA-256 digest, and create a unique payload file with exclusive-create semantics.
- Include an envelope marker/version, owner durable execution ARN, owner entity/operation ID, payload type, file path, and content digest.
- On deserialization, validate the recognized envelope, expected execution directory and filename, base-path containment, symbolic links, owner rules, and content digest.
- Define explicit rules for safe cross-execution references such as invocation input/results. When an exception is forwarded through a child context, deserialize and reserialize it under the new owner rather than copying an owner-bound file envelope.
- Preserve compatibility by continuing to read the existing legacy envelope format for an appropriate migration period.
- Document storage lifecycle guidance because immutable payload publication can create orphaned files after checkpoint replacement.
- Extend the cloud coverage tracked by #527 with immutable publication and ownership/integrity scenarios.
Is this a breaking change?
No. Existing constructors and configuration should remain compatible, and legacy envelopes can continue to be readable.
Does this require an RFC?
Yes.
Additional Context
The Java SDK filesystem SerDes work in aws/aws-durable-execution-sdk-java#648 uses immutable publication, versioned ownership metadata, content hashes, containment checks, and symbolic-link rejection. This issue requests equivalent guarantees adapted to Python's synchronous SerDes[Any] and configurable inner value SerDes.
- Dominant language
- Python
- Stars
- 59
- Forks
- 32
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 76
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aws/aws-durable-execution-sdk-python
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
aws/aws-durable-execution-sdk-python#977 ·
Maintainers usually reply within 1 day
-
[AgentCore session] Remove the unused session timeoutPossibly taken @srivilliamsai claimed this today. Openneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
aws/aws-durable-execution-sdk-python#970 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws/aws-durable-execution-sdk-python#952 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W4: Document sessions in the worker READMEPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 1/5 1-3 hours Newbie friendliness 76/100
aws/aws-durable-execution-sdk-python#950 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W3: Add the session ID and job count to worker logsPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
aws/aws-durable-execution-sdk-python#949 ·
Maintainers usually reply within 1 day
All issues in aws/aws-durable-execution-sdk-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Unused loot bag group names left in dreamcraft en_US.langPossibly taken A pull request linked to this issue is open or already merged. OpenStatus: Triage Suggestion
Difficulty 1/5 Under an hour Newbie friendliness 90/100
GTNewHorizons/GT-New-Horizons-Modpack#27789 ·
Maintainers usually reply within 1 day
-
bug frontend sev-5
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
-
core good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
b4b code health next
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ESCOMP/CTSM#4330 · 1 comment ·
Maintainers usually reply within 1 day