FileResultStore.delete_expired crashes on JSON-materialized cache results

Abierto Apto para principiantes
#1,813 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
75/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend, testing

Línea de trabajo

Empieza por hamilton/caching/stores/file.py, especialmente por FileResultStore.delete_expired() y _load_result_from_path(), y después inspecciona las pruebas del almacén de archivos para comprobar la cobertura de la limpieza. Ejecuta las pruebas específicas y añade un caso de regresión usando JSONDataSaver y JSONDataLoader; el trabajo estará terminado cuando la limpieza se complete y el resultado no vencido siga siendo legible.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Current behavior

FileResultStore.delete_expired() raises UnpicklingError after storing a result with the built-in JSON saver/loader. The result can be read successfully before cleanup, and nothing has expired.

This prevents cleanup of a store containing valid JSON-materialized results.

Steps to reproduce

from pathlib import Path
from tempfile import TemporaryDirectory

from hamilton.caching.stores.file import FileResultStore
from hamilton.io.default_data_loaders import JSONDataLoader, JSONDataSaver

with TemporaryDirectory() as directory:
    store = FileResultStore(directory)
    store.set(
        "example",
        {"value": 1},
        saver_cls=JSONDataSaver,
        loader_cls=JSONDataLoader,
    )
    assert store.get("example") == {"value": 1}
    print(sorted(path.name for path in Path(directory).iterdir()), flush=True)
    store.delete_expired()
    assert store.get("example") == {"value": 1}

Observed output and traceback (paths shortened):

['example', 'example.json']
Traceback (most recent call last):
  File "repro.py", line 17, in <module>
    store.delete_expired()
  File "hamilton/caching/stores/file.py", line 138, in delete_expired
    stored_result = self._load_result_from_path(file_path)
  File "hamilton/caching/stores/file.py", line 55, in _load_result_from_path
    return StoredResult.load(data)
  File "hamilton/caching/stores/base.py", line 234, in load
    loaded = pickle.loads(raw)
_pickle.UnpicklingError: invalid load key, '{'.

Expected behavior

Cleanup should complete without attempting to unpickle the JSON payload, and the unexpired result should remain readable.

Library and system information

  • Hamilton 1.90.0, editable checkout at 2bd9dbc7ee22e1c2a4f6cd687a353aa0079d4d81
  • Python 3.11.17, macOS arm64
  • Compared both affected files (caching/stores/file.py and caching/stores/base.py) with upstream main at ef88b1e96140a01f3d72967316c2335f8b62a067; they are byte-for-byte identical.

Proposed contribution

set() writes a pickled loader reference and a separate .json payload into the same directory. delete_expired() iterates every directory entry and passes each to StoredResult.load(), including the JSON payload.

I would like to contribute a focused fix that distinguishes stored-result entries from materialized payloads, with a regression test showing cleanup preserves a readable JSON result. Would you be open to me taking this on, and is there a preferred way to identify those entries? I would keep any broader expiration/serialization redesign separate.

Lenguaje dominante
Jupyter Notebook
Estrellas
2.6k
Forks
230
Merge medio
3 d 3 h
PR fusionados (30 d)
22

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/hamilton

Todos los issues de apache/hamilton

Issues similares

Más issues de Backend & API Design

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.