[Feature]: Add composable SerDes transformation pipelines
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
Research direction
Start by reading the existing SerDes and FileSystemSerDes implementations and their configuration APIs to understand the current value-codec boundary. Define the stage protocol and composition semantics through the required RFC, then add tests covering ordering, pass-through input, null and empty values, malformed recognized envelopes, and custom value types while preserving backward compatibility.
Written by the indexing model from the issue text.
Description
What would you like?
Add a composable SerDes processing pipeline while preserving the existing object-to-string SerDes[T] API as the value-codec boundary.
Today, transformations such as filesystem offloading, compression, encryption, signing, or custom envelopes must each wrap and own another complete SerDes. This makes independent transformations difficult to reuse or reorder.
The proposed model is:
value -> root SerDes -> string stage -> string stage -> checkpoint
value <- root SerDes <- string stage <- string stage <- checkpoint
Possible Implementation
- Introduce a synchronous
SerDesStageprotocol whose serialize/deserialize methods accept and returnstrplusSerDesContext. - Add a
ComposableSerDesor composition helper that starts with one existingSerDes[T], applies stages in forward order during serialization, and reverses them during deserialization. - Require stages to emit a self-identifying/versioned format. During deserialization, a stage should:
- reverse valid input in its own format;
- raise
SerDesErrorfor malformed input that identifies itself as that stage; - return unrecognized input unchanged.
- Keep existing
SerDesimplementations and configuration APIs backward compatible. - Allow
FileSystemSerDesfunctionality to be exposed as a stage so callers can choose any root value codec and can place compression, encryption, or signing before or after storage as appropriate. - Add tests for stage order, pass-through of external invocation input, null/empty values, malformed recognized envelopes, and custom value types.
Is this a breaking change?
No.
Does this require an RFC?
Yes.
Additional Context
Python's current FileSystemSerDes already supports a configurable inner value SerDes. A pipeline would separate that value-codec responsibility from reusable string transformations instead of adding another nested-wrapper layer. A comparable Java design is being developed in aws/aws-durable-execution-sdk-java#648.
- Dominant language
- Python
- Stars
- 59
- Forks
- 32
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 76
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aws/aws-durable-execution-sdk-python
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
aws/aws-durable-execution-sdk-python#977 ·
Maintainers usually reply within 1 day
-
[AgentCore session] Remove the unused session timeoutPossibly taken @srivilliamsai claimed this today. Openneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
aws/aws-durable-execution-sdk-python#970 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws/aws-durable-execution-sdk-python#952 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W4: Document sessions in the worker READMEPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 1/5 1-3 hours Newbie friendliness 76/100
aws/aws-durable-execution-sdk-python#950 ·
Maintainers usually reply within 1 day
-
[AgentCore session] W3: Add the session ID and job count to worker logsPossibly taken @ParidelPooya claimed this 3 days ago. Openenhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
aws/aws-durable-execution-sdk-python#949 ·
Maintainers usually reply within 1 day
All issues in aws/aws-durable-execution-sdk-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Unused loot bag group names left in dreamcraft en_US.langPossibly taken A pull request linked to this issue is open or already merged. OpenStatus: Triage Suggestion
Difficulty 1/5 Under an hour Newbie friendliness 90/100
GTNewHorizons/GT-New-Horizons-Modpack#27789 ·
Maintainers usually reply within 1 day
-
bug frontend sev-5
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
-
core good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
b4b code health next
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ESCOMP/CTSM#4330 · 1 comment ·
Maintainers usually reply within 1 day