Memory growth in timeframe builder when merging more frames
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
Research direction
The issue names src/HepMC3DataSource.cc and HepMC3DataSource::openNextFile() as the affected entry point. Read that function and inspect how it replaces reader_; then run the existing tests that cover reopening HepMC3 input files. Done means the old reader is closed before replacement and the output remains identical while memory growth is avoided.
Written by the indexing model from the issue text.
Description
Cause:
- Reopen without close: when a background file runs out and has to repeat, TFB 0.9.2 opens it again (HepMC3DataSource::openNextFile()). It
replaces the old reader without calling close() on it. - Nothing cleans up: HepMC3 3.3.0's ReaderRootTree has no destructor, so a reader dropped that way keeps its ROOT file, tree and read buffers
in memory. - SYNRAD reopens constantly: the file holds only about 31 frames of photons, so each job leaks memory every ~31 frames.
Evidence:
- Isolated test: opening and dropping the SYNRAD reader 5 times took memory from 514 MB to 1,296 MB. Calling close() first kept it flat at
about 524 MB. - Patched job test:
| 500 frames | peak memory |
|---|---|
| stock TFB | 7.7 GB (all 20 jobs went over 6 GB) |
| patched TFB | 1.9 GB, flat (3 jobs, all within the 6 GB request) |
| SignalBackgroundMerger | 2.0 GB |
The patched output is identical to stock TFB: all 42 summary arrays match exactly in each of the 3 jobs. So the fix changes memory and
nothing else.
Fix: close the old reader before opening the new one, 7 added lines in src/HepMC3DataSource.cc:
if (reader_) {
reader_->close();
reader_.reset();
}
reader_ = std::make_sharedHepMC3::ReaderRootTree(input_file);
- Dominant language
- C++
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from eic/TimeframeBuilder
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
eic/TimeframeBuilder#98 ·
-
Consider a renameOpen
Difficulty 5/5 Over a week Newbie friendliness 25/100
eic/TimeframeBuilder#88 · 9 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
eic/TimeframeBuilder#81 ·
-
Produce a JANA2 plugin where this is the JEventSourcePossibly taken @simonge claimed this 273 days ago. Open
eic/TimeframeBuilder#68 · 1 reaction · 2 assignees ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
eic/TimeframeBuilder#58 ·
All issues in eic/TimeframeBuilder
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
mw00/project-maya#877 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
llvm/offload-test-suite#1703 ·
Maintainers usually reply within 1 day
-
backlog bug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
EchoTools/nevr-runtime#2154 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
BGI-shenzhen/PopLDdecay#25 ·