[Bug] File descriptor leak in CLI CSV import: CSVParser from readCsvFile is never closed
还没有人认领这个 Issue。
评估
调研方向
首先阅读 AbstractDataTool.readCsvFile 以及指定的导入入口:ImportData.importFromSingleFile、ImportDataTree.importFromCsvFile、ImportDataTable.importFromCsvFile 和 ImportSchemaTree.importSchemaFromCsvFile。在较低的 ulimit 下使用许多小型 CSV 文件重现该问题,然后验证每个解析器在正常完成和提前 return 时都会被关闭,且不会耗尽文件描述符。
由索引模型根据 Issue 内容生成。
描述
Search before asking
- I searched in the issues and found nothing similar.
Version
master (2.0.x). The affected code is also present in released 2.0.x versions.
Describe the bug and provide the minimal reproduce step
AbstractDataTool.readCsvFile(String) builds a CSVParser over new InputStreamReader(new FileInputStream(path)) and returns it. The CSVParser owns that FileInputStream, but the CLI import code paths that call it never close the returned parser — it is assigned to a local inside a plain try { ... } block (no try-with-resources, no finally). Every imported file therefore leaks its file descriptor, and the early returns for an empty file or an invalid header leak it immediately, because the parser is opened before those checks run.
Affected call sites (current master, also present in released 2.0.x):
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportData.java—importFromSingleFileiotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportDataTree.java—importFromCsvFileiotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportDataTable.java—importFromCsvFileiotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ImportSchemaTree.java—importSchemaFromCsvFile(this class has its own copy ofreadCsvFile)
Minimal reproduce step:
- Create a directory containing a large number of small CSV files — more than the process open-file limit (for example a few thousand files under
ulimit -n 1024). - Run the CLI data import over that directory.
- The import fails partway through with
Too many open files. (A single import already leaks one descriptor; it is simply not fatal until enough accumulate.)
What did you expect to see?
Each CSVParser (and the FileInputStream it wraps) is closed after the file is processed, on every exit path — including the empty-file / invalid-header early returns. Importing a large directory of CSV files should not exhaust the process's file descriptors.
What did you see instead?
The CSVParser returned by readCsvFile is never closed, so its underlying FileInputStream stays open. Importing a directory with enough CSV files leaks descriptors until the import fails with Too many open files.
Anything else?
The record Stream is fully consumed inside the same block before the method returns, so consuming each parser in a try-with-resources (closing it on scope exit) is safe.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- 主要语言
- Java
- 星标
- 6.4k
- 派生
- 1.2k
- 平均合并
- 1 天 17 小时
- 30 天内合并 PR
- 152
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/iotdb 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 82/100
-
IoTDB Edge: stop-edge.sh does not stop its own process when IOTDB_HOME is set, and reports success 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 78/100
相似的 Issue
-
area-deployment area-integrations triage:bot-seen
难度 2/5 半天 新手友好度 86/100
-
难度 2/5 1-3 小时 新手友好度 75/100
apache/flink-agents#1156 ·
-
area/connectors autoteam community connectors/source/shopify needs-triage team/use type/bug
难度 2/5 1-3 小时 新手友好度 84/100
-
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 1/5 1 小时以内 新手友好度 85/100