How to recursively unstructure with hooks?

Start with the reproducer using cattrs 23.2.3, attrs, JsonConverter, and configure_tagged_union. Read the registered Frame unstructure hook alongside register_unstructure_hook_factory and unstructure_attrs_asdict to trace why nested omission differs. Done means the intended None-valued field is omitted while the added Frame field and tagged-union output remain present.

• cattrs version: 23.2.3
• Python version: 3.11
• Operating System: Windows (dev)/Linux (prod)

Hey there,

I have put together this small example to illustrate my issue. Context is JSON serialization.

There is an outer class Frame, whose data field is a UnionType (used as tagged union later).
Some fields have a None as default value -- those should not appear in the resulting JSON string:

import attrs
from cattrs.gen import make_dict_unstructure_fn
from cattrs.preconf.json import JsonConverter, make_converter
from cattrs.strategies import configure_tagged_union
from types import UnionType
from typing import get_origin, get_args

@attrs.frozen
class A:
    to_keep: str = 'foo'
    to_skip: str|None = None

@attrs.frozen
class B:
    to_keep: str = 'bar'
    to_skip: str|None = None

FrameData = dict | A | B

@attrs.frozen
class Frame:
    data: FrameData
    to_skip: str|None = None

Then there are two helpers and a hook factory:

def _contains_nonetype(type_) -> bool:
    if get_origin(type_) is UnionType:
        return type(None) in get_args(type_)
    return type_ is type(None)

def _is_attrs_with_none_defaults(type_: type) -> bool:
    return attrs.has(type_) and any(_contains_nonetype(a.type) and a.default is None
                                    for a in attrs.fields(type_))

def _get_unstructure_without_nones(cls):
        unstructure = make_dict_unstructure_fn(cl=cls, converter=conv)
        fields = [a.name for a in attrs.fields(cls) if _contains_nonetype(a.type) and a.default is None]

        def unstructure_without_nones(obj):
            unstructured = unstructure(obj)
            for field in fields:
                if unstructured[field] is None:
                    unstructured.pop(field)
            return unstructured

        return unstructure_without_nones

Then there is the converter, an additional hook (to add something special to the Frame object) and the tagged union definition:

conv = make_converter()
conv.register_unstructure_hook_factory(predicate=_is_attrs_with_none_defaults, factory=_get_unstructure_without_nones)
conv.register_unstructure_hook(Frame, lambda obj: {'to_add': 'something special', **conv.unstructure_attrs_asdict(obj)})
configure_tagged_union(union=FrameData, converter=conv, tag_name='_type', tag_generator=lambda t: t.__name__.casefold(), default=dict)

If I serialize the Frame instance:

print(conv.dumps(Frame(data=A())))

I get this:

{"to_add": "something special", "data": {"to_keep": "foo", "_type": "a"}, "to_skip": null}

But what I need is this (with the None-value attribute omitted):

{"to_add": "something special", "data": {"to_keep": "foo", "_type": "a"}}

I am sure it is related to this hook, because it works as expected without it:

conv.register_unstructure_hook(Frame, lambda obj: {'to_add': 'something special', **conv.unstructure_attrs_asdict(obj)})

It would be great if you could help me out again