Bug: data masking prepare_data leaves dataclasses and Pydantic models unconverted

Offen
  • Stimmt etwas nicht?
#8,672 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@therealemmanueloppong arbeitet bereits daran.

Seit 01.11.2026.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Anfängerfreundlichkeit
57/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
python
Bereich
backend

Rechercherichtung

Start with prepare_data in aws_lambda_powertools.utilities.data_masking.base and inspect how _visited is updated during recursive conversion. Run the reproduction in the issue and the existing circular-reference test, then add regression coverage in tests/functional/data_masking/_pydantic/test_data_masking_with_pydantic.py. Done means repeated dataclass and Pydantic instances are converted while circular references remain detected.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Expected Behaviour

DataMasking.erase (and the other operations that go through prepare_data) should convert every dataclass and Pydantic model in the input to plain data, however many there are and whether or not the same instance appears twice.

Current Behaviour

prepare_data records the id() of every object it has visited in _visited to detect circular references, and never removes them. That includes the temporary dicts built by dataclasses.asdict() and model_dump(). Once one of those is freed, CPython reuses its id for a later object, which is then taken for a circular reference and returned unconverted. An instance referenced twice is skipped the same way. erase then fails with TypeError: Object of type X is not JSON serializable.

Code snippet
from dataclasses import dataclass
from pydantic import BaseModel
from aws_lambda_powertools.utilities.data_masking import DataMasking
from aws_lambda_powertools.utilities.data_masking.base import prepare_data

class Card(BaseModel):
    number: str

@dataclass
class Item:
    card: Card

print(prepare_data({"items": [Item(Card(number=str(i))) for i in range(5)]}))
# some items still contain Card(...) objects instead of dicts

@dataclass
class Person:
    age: int

p = Person(age=30)
DataMasking().erase({"items": [p, p]}, fields=["items[*].age"])
# TypeError: Object of type Person is not JSON serializable
Possible Solution

Keep in _visited only the objects on the current conversion path (add before recursing, remove in a finally). Circular references are still caught, and the existing circular-reference test still passes. I have this fix with regression tests in tests/functional/data_masking/_pydantic/test_data_masking_with_pydantic.py (fail on develop, pass with the change) and can open a PR once this is triaged.

Steps to Reproduce

Run the snippet above on develop (5785377).

Powertools for AWS Lambda (Python) version

latest (3.37.0, develop)

AWS Lambda function runtime

3.13

Packaging format used

PyPi

Debugging logs
N/A
Vorherrschende Sprache
Python
Sterne
3.3k
Forks
511
Ø Merge
3 T. 4 Std.
Gemergte PRs (30 T.)
98

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus aws-powertools/powertools-lambda-python

Alle Issues in aws-powertools/powertools-lambda-python

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.