[python] Verify generated Python serialization against pydsdl's built-in serializer

Open
#416 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
testing

Research direction

The verification suite lives in verification/python; start from suite/test_rand.py and suite/util.py:make_random_object to see how random objects are built. Add cross-checks using pydsdl.serialize/deserialize in the default test session, normalizing values with suite/util.py:are_close. Done means checks 1–3 pass against pydsdl ≥1.25.0 and dependency pin updated.

Written by the indexing model from the issue text.

Description

build and test Python

Summary

pydsdl 1.25.0 added pydsdl.serialize(schema, obj) and pydsdl.deserialize(schema, data) (OpenCyphal/pydsdl#120, #121). They encode and decode a DSDL type directly from its pydsdl.CompositeType, without generated code. That makes them an implementation of the DSDL wire format that is independent of Nunavut. The Python verification suite should use them as a reference to check the generated Python code against.

Why

Today every serialization check in verification/python compares the generated code with itself:

  • suite/test_rand.py serializes random objects and checks that they deserialize back to the same object.
  • suite/test_fuzz.py and tools/fuzz_deserialize.py check that deserialize() never raises and that whatever it returns survives a round trip.

A round trip can't detect a fault that the serializer and the deserializer share. If both get a union tag width, bit order, padding, an implicit truncation or extension rule, or a delimiter header wrong in the same way, every current test still passes and the wire format is still wrong. Comparing against a second implementation catches that class of fault.

Proposal

For every non-service type the suite already generates (and the request and response types of services), using the random objects the suite already builds (suite/util.py:make_random_object):

  1. Encoding agrees: b"".join(nunavut_support.serialize(obj)) equals pydsdl.serialize(model, to_builtin(obj)), byte for byte.
  2. Nunavut decodes pydsdl's output: nunavut_support.deserialize(cls, [memoryview(pydsdl.serialize(model, ref))]) equals the reference object ref.
  3. pydsdl decodes Nunavut's output: pydsdl.deserialize(model, nunavut_bytes) equals to_builtin(obj), after the normalization below.
  4. Both reject the same input (optional, fuzz-driven): for mutated byte strings, Nunavut's deserialize() returns None if and only if pydsdl.deserialize() raises pydsdl.SerDesError. If both accept the input, their results agree.

Checks 1–3 belong in the default test session. Check 4 could reuse the Hypothesis mutators in suite/fuzzing.py, with the same example budget.

Things to settle in the design

  • Value representation. to_builtin and pydsdl don't use exactly the same Python values. For example, pydsdl returns str for UTF-8 arrays and bytes for byte arrays. Float16 and float32 values need rounding-aware comparison, and NaN needs special handling (suite/util.py:are_close already does some of this). The comparison should normalize both sides explicitly, not loosen equality in general.
  • Delimited types. The top-level serialization of a delimited type has no delimiter header in either implementation (with_delimiter_header=False is pydsdl's default). Nested delimited fields carry a header in both. The tests should cover both cases on purpose.
  • Disagreements. pydsdl's serializer is new. When the two implementations disagree, the DSDL specification decides which one is wrong. Any fault found in pydsdl should be reported upstream rather than worked around in Nunavut.
  • Cost. pydsdl's serializer is pure Python and works bit by bit. The suite's existing size limit (_MAX_EXTENT_BYTES) and sample count should keep run time bounded. The suite's performance timing should leave pydsdl's time out.

Prerequisite

This needs pydsdl ≥ 1.25.0 in the verification environment, which means raising Nunavut's pydsdl dependency and moving the submodules/pydsdl pin to 1.25.0.

Out of scope

The C and C++ suites could use the same reference through test vectors generated in Python at build time. That should be a follow-up once the Python comparison has shaken out disagreements.

Dominant language
Python
Stars
48
Forks
26
Avg merge
1d 18h
Merged PRs (30d)
9

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OpenCyphal/nunavut

All issues in OpenCyphal/nunavut

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.