Unbounded cache in SpecValidator.iter_errors retains validator instances and schemas
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 55/100
- Type d'issue
- Bug
- Clarté
- Plutôt claire
- Activité
- Active
- Stack technique
- python
- Domaine
- performance
Piste de recherche
Start in openapi_spec_validator/validation/validators.py around SpecValidator.iter_errors at lines 79-86, and run the minimal reproduction to confirm cache growth and release after cache_clear(). Trace how validate() creates validators and how repeated iteration is preserved. Done means discarded specifications and validator instances can be collected without unbounded historical retention, while repeated iteration still works.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Bug description
Repeated calls to validate() permanently retain each validator instance and its schema, even after validation succeeds, the caller discards the schema, and gc.collect() runs. This causes approximately linear memory growth in long-running applications that repeatedly load/validate specifications.
Reproduced with openapi-spec-validator 0.9.0, CPython 3.11.15, macOS arm64. The same retention mechanism was also reproduced with 0.8.5. The reproduction below uses only this package and the Python standard library; no Prance, web server, application code, or external data is needed.
Minimal reproduction
Install openapi-spec-validator==0.9.0 and run in a fresh process:
import gc
import tracemalloc
from importlib.metadata import version
from openapi_spec_validator import validate
from openapi_spec_validator.validation.validators import SpecValidator
def make_spec():
return {
'openapi': '3.0.0',
'info': {'title': 'Memory reproduction', 'version': '1.0.0'},
'paths': {},
'x-payload': 'x' * 65536,
}
# Warm up lazy imports; clearing the private cache is diagnostic only.
validate(make_spec())
cache = SpecValidator.iter_errors.__wrapped__
cache.cache_clear()
gc.collect()
tracemalloc.start()
print('openapi-spec-validator', version('openapi-spec-validator'))
for count in (50, 100, 150):
for _ in range(50):
validate(make_spec())
gc.collect()
print(count, 'cached validators:', cache.cache_info().currsize,
'retained bytes:', tracemalloc.get_traced_memory()[0])
cache.cache_clear()
gc.collect()
print('after cache_clear:', tracemalloc.get_traced_memory()[0])
tracemalloc.stop()
Observed output (exact byte counts may vary):
openapi-spec-validator 0.9.0
50 cached validators: 50 retained bytes: 3498052
100 cached validators: 100 retained bytes: 6960377
150 cached validators: 150 retained bytes: 10417844
after cache_clear: 23183
Each input is a freshly allocated but content-identical valid specification, containing a synthetic 64 KiB extension value. No input or validation result is retained by the caller. The memory values are current Python allocations measured by tracemalloc after GC, not peak RSS.
Expected behavior
Once validation finishes and the caller releases the specification, the validator and its schema should be eligible for garbage collection. Repeated validation should not create an unbounded process-wide collection of historical validator instances.
Suspected cause
SpecValidator.iter_errors is decorated with lru_cache(maxsize=None). Since the cache key includes self, every instance created by the validate() shortcut remains reachable from the class-level cache, together with its schema. This happens even for successful validations with no errors.
Clearing this private cache releases almost all of the retained memory in the experiment. The __wrapped__/cache_clear() calls above are diagnostic controls, not a proposed application workaround: globally clearing the cache can interfere with unrelated callers.
Could the cached iterable be owned by the validator instance, or otherwise avoid globally retaining validator instances, while preserving repeated iteration behavior?
I searched existing issues/PRs for memory, leak, lru_cache, and iter_errors but did not find an equivalent report.
- Langage dominant
- Python
- Étoiles
- 409
- Forks
- 73
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Guide de contribution
Ouvrir le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de python-openapi/openapi-spec-validator
-
Difficulté 3/5 1-2 jours Accessibilité débutants 55/100
-
Difficulté 3/5 1-2 jours Accessibilité débutants 30/100
-
Difficulté 4/5 3-5 jours Accessibilité débutants 42/100
python-openapi/openapi-spec-validator#400 · 1 commentaire ·
-
kind/bug/confirmed
Difficulté 5/5 Plus d'une semaine Accessibilité débutants 25/100
python-openapi/openapi-spec-validator#373 · 1 commentaire ·
-
Difficulté 3/5 1-2 jours Accessibilité débutants 45/100
Toutes les issues de python-openapi/openapi-spec-validator
Issues similaires
-
[Bug] reef-hermes tells me to resume with hermes --resume, which does not work from my shell Ouvertearea: harness bug status: needs-triage
Difficulté 2/5 1-3 heures Accessibilité débutants 75/100
Human-Agent-Society/reef#625 ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 70/100
-
Difficulté 1/5 Moins d'une heure Accessibilité débutants 80/100
learningequality/kolibri#15351 · 2 commentaires ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 75/100
-
Name consistency Ouverte
Difficulté 2/5 1-3 heures Accessibilité débutants 75/100
eellak/triplestore#65 · 1 commentaire ·