Memory leak: the shared `validation_aborted_error` keeps every traceback it is raised through
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in graphql/validation/validate.py, where the shared validation_aborted_error is raised, and read how validate() handles reaching max_errors. The reproduction in the issue demonstrates the traceback and memory growth; run it before and after the change to confirm repeated calls no longer retain growing tracebacks. Done when each validation call raises its own error and the memory growth levels off.
Written by the indexing model from the issue text.
Description
Since aab6d50 (first released in 3.3.0a3), validate() raises one shared, module-level ValidationAbortedError once max_errors is reached. When an existing exception object is raised again, Python adds the new frames to its existing __traceback__. So every aborted validation lengthens the singleton's traceback, and that traceback keeps those frames, and with them each request's document and validation context, alive for the life of the process.
Any client can trigger this by sending a query with more than 100 validation errors. In a long-running server, memory grows with every such request.
Reproduction (graphql-core 3.3.0, CPython 3.12):
import gc, tracemalloc
from graphql import build_schema, parse, validate
from graphql.validation.validate import validation_aborted_error
schema = build_schema("type Query { a: Int }")
document = parse("{ " + " ".join(f"f{i}" for i in range(101)) + " }")
tracemalloc.start()
validate(schema, document)
gc.collect()
base = tracemalloc.get_traced_memory()[0]
for _ in range(200):
validate(schema, document)
gc.collect()
tb, n = validation_aborted_error.__traceback__, 0
while tb:
n, tb = n + 1, tb.tb_next
print(n, (tracemalloc.get_traced_memory()[0] - base) // 1024, "KiB")
Result:
| Repeated calls | Traceback entries | Memory retained |
|---|---|---|
| 1 | 16 | 147 KiB |
| 10 | 88 | 1,389 KiB |
| 100 | 808 | 12,549 KiB |
| 200 | 1,608 | 24,820 KiB |
That is about 124 KiB more per aborted validation, with no upper limit. The same script on 3.2.13, which creates a new error each time, levels off at about 259 KiB.
Suggested fix: create a new ValidationAbortedError on each validate() call instead of reusing the module-level instance. Clearing __traceback__ before each raise would also work, but every caller's error list would still share one mutable error object.
- Dominant language
- Python
- Stars
- 532
- Forks
- 154
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 6
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from graphql-python/graphql-core
-
`collect_subfields` memoization keyed on `id()` of `FieldDetails` that are not kept alive returns another field's subfieldsPossibly taken @jadenfix claimed this 7 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 72/100
graphql-python/graphql-core#284 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 35/100
graphql-python/graphql-core#278 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
graphql-python/graphql-core#257 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
graphql-python/graphql-core#223 · 1 comment ·
-
documentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
graphql-python/graphql-core#220 · 4 comments ·
All issues in graphql-python/graphql-core
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Arekkazu/sgpmp-backend#744 ·
Maintainers usually reply within 1 day
-
agent/reviewer
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectbluefin/utah-packages#634 ·
Maintainers usually reply within 1 day
-
enhancement good first issue help wanted
Difficulty 2/5 1-2 days Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
home-assistant/core#188613 ·
Maintainers usually reply within 1 day
-
WebsiteOpengood first issue
Difficulty 2/5 1-3 hours Newbie friendliness 91/100
TanbirRamim/open-source-radar#314 ·
Maintainers usually reply within 1 day