Bug: data masking prepare_data leaves dataclasses and Pydantic models unconverted

Aperta
  • Qualcosa non va?
#8,672 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
57/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend

Direzione di ricerca

Start with prepare_data in aws_lambda_powertools.utilities.data_masking.base and inspect how _visited is updated during recursive conversion. Run the reproduction in the issue and the existing circular-reference test, then add regression coverage in tests/functional/data_masking/_pydantic/test_data_masking_with_pydantic.py. Done means repeated dataclass and Pydantic instances are converted while circular references remain detected.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Expected Behaviour

DataMasking.erase (and the other operations that go through prepare_data) should convert every dataclass and Pydantic model in the input to plain data, however many there are and whether or not the same instance appears twice.

Current Behaviour

prepare_data records the id() of every object it has visited in _visited to detect circular references, and never removes them. That includes the temporary dicts built by dataclasses.asdict() and model_dump(). Once one of those is freed, CPython reuses its id for a later object, which is then taken for a circular reference and returned unconverted. An instance referenced twice is skipped the same way. erase then fails with TypeError: Object of type X is not JSON serializable.

Code snippet
from dataclasses import dataclass
from pydantic import BaseModel
from aws_lambda_powertools.utilities.data_masking import DataMasking
from aws_lambda_powertools.utilities.data_masking.base import prepare_data

class Card(BaseModel):
    number: str

@dataclass
class Item:
    card: Card

print(prepare_data({"items": [Item(Card(number=str(i))) for i in range(5)]}))
# some items still contain Card(...) objects instead of dicts

@dataclass
class Person:
    age: int

p = Person(age=30)
DataMasking().erase({"items": [p, p]}, fields=["items[*].age"])
# TypeError: Object of type Person is not JSON serializable
Possible Solution

Keep in _visited only the objects on the current conversion path (add before recursing, remove in a finally). Circular references are still caught, and the existing circular-reference test still passes. I have this fix with regression tests in tests/functional/data_masking/_pydantic/test_data_masking_with_pydantic.py (fail on develop, pass with the change) and can open a PR once this is triaged.

Steps to Reproduce

Run the snippet above on develop (5785377).

Powertools for AWS Lambda (Python) version

latest (3.37.0, develop)

AWS Lambda function runtime

3.13

Packaging format used

PyPi

Debugging logs
N/A
Lingua principale
Python
Stelle
3.3k
Fork
511
Merge medio
3g 12h
PR unite (30g)
93

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di aws-powertools/powertools-lambda-python

Tutte le issue di aws-powertools/powertools-lambda-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.