RDataFrame in MultiThread with RNTuples
维护者通常 1 天内回复
@jblomer 已经在做这个了。
开始于 2026年8月24日。
评估
这个 Issue 还没有评估数据。
描述
Check duplicate issues.
- Checked for duplicates
Description
Dear ROOT developers,
We have an analysis framework for CMS data that uses ROOT RDataFrame in MultiThread. In view of the expected adoption of RNTuples, we tried to use an RDataFrame script to convert CMS data stored in ROOT files as flat ntuples (NanoAOD, TTree structure) to RNTuples, which we can then read with our usual framework.
However, when running on the RNTuples the program opens a number of files equal to the number of available threads, waits for all of these files to be processed, and only then opens the next batch. As a result, the execution is limited by the slowest file-thread combination of the batch.
We have prepared a reproducer [linked below with instructions and comments] that generates a set of files as RNTuples, processes the events, and reproduces the issue. We have observed the same behavior on lxplus, on our institute's cluster and our laptop.
On lxplus and our institute cluster we also get the warning
Warning in <RIoUring>: io_uring is unexpectedly not available because:
Error initializing io_uring: Operation not permitted
Warning in <RRawFileUnix>: io_uring setup failed, falling back to blocking I/O in ReadV
Thank you in advance for your help!
Francesco Orlandi and Federico Ferri (@ferriff)
Reproducer
On lxplus:
- copy
reproduce_problem.ccandj.jsonin a folder - compile:
clang++ reproduce_problem.cc $(root-config --cflags --libs) -o reproduce_problem - execute:
./reproduce_problem -g -n 3 -j j.json
Options:
g(activate generation of files, necessary only the first time)n(number of threads for MT execution)j(json file name containing the input files name)
The program generates 100 files: the odd-numbered files contain 9 entries, while the even-numbered files contain 10000 entries. During the execution, we use CPU time by performing some mathematical computations for each entry. For each “batch”, we print the file names and the number of entries contained in each file. Together with the progress bar from RDataFrame, this shows the problem.
ROOT version
6.40.02, built on Jun 20 2026
Installation method
lxplus
Operating system
Linux
Additional context
No response
- 主要语言
- C++
- 星标
- 3.3k
- 派生
- 1.6k
- 平均合并
- 2 天 16 小时
- 30 天内合并 PR
- 194
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
root-project/root 的其他 Issue
-
thisroot.sh: wrong ROOTSYS in zsh when the shell's command line has a newline可能已有人在做 @dpiparo 于 3 天前认领。 未关闭bug in:Build System
难度 2/5 1-3 小时 新手友好度 86/100
root-project/root#23746 · 已指派 1 人 ·
维护者通常 1 天内回复
-
[tmva][sofie] Inputs of unsupported type (e.g. float16) give code that does not compile instead of an error可能已有人在做 @lmoneta 于 12 天前认领。 未关闭AI-Assisted bug in:SOFIE
难度 2/5 1-3 小时 新手友好度 65/100
root-project/root#23655 · 已指派 1 人 ·
维护者通常 1 天内回复
-
bug
难度 3/5 1-2 天 新手友好度 71/100
root-project/root#23790 ·
维护者通常 1 天内回复
-
bug in:Core Libraries
难度 4/5 3-5 天 新手友好度 48/100
root-project/root#23777 ·
维护者通常 1 天内回复
-
`TChain::GetEntries` internal caching ineffective?可能已有人在做 @pcanal 于 5 天前认领。 未关闭improvement in:TTree
root-project/root#23738 · 1 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
查看 root-project/root 的全部 Issue
相似的 Issue
-
enhancement GUI maintainer-review
难度 2/5 1-3 小时 新手友好度 66/100
aethersdr/AetherSDR#6681 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
维护者通常 1 天内回复
-
Bug
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
area: ui bug good first issue P2: medium value: medium
难度 2/5 1-3 小时 新手友好度 75/100
patrick-hudson/put.gd-desktop#366 ·
维护者通常 1 天内回复