[Bug] Compaction fails with Windows ERROR_NO_SYSTEM_RESOURCES while Pipe workers wait for memory to parse TsFiles (2.0.8)

The bug occurs in the compaction and pipe subsystems under concurrent load. Start by examining the classes mentioned: RewriteCrossSpaceCompactionSelector, LocalTsFileInput, and PipeTsFileInsertionEvent. Look at how TsFile parsing memory is managed and how file reads are coordinated. The goal is to make compaction and pipe tasks back off when system resources are low, preventing the Windows ERROR_NO_SYSTEM_RESOURCES. Check for existing concurrency controls or rate-limiting mechanisms.

Search before asking

• I searched in the issues and found nothing similar.

Version

On IoTDB 2.0.8 (Windows, all-bin)

Describe the bug and provide the minimal reproduce step

On IoTDB 2.0.8 (Windows, all-bin), three Pipe processor threads concurrently wait tens of seconds for memory to parse sequence TsFiles. At the same time, compaction is running on the same databases. Cross-space compaction then fails while reading an older TsFile: the native FileChannel.read returns Windows error 1450, ERROR_NO_SYSTEM_RESOURCES (系统资源不足，无法完成请求的服务。).
This is not a Java heap OutOfMemoryError. The read fails inside sun.nio.ch.FileDispatcherImpl.read0.
2.0.10 PR #17542 ("Do not stop pipe for Waited for memory to parse TsFile") only keeps the Pipe subtask from stopping on that timeout. It does not address the native read failure under the same concurrent Pipe + compaction load.

What did you expect to see?

Pipe TsFile parsing and compaction should back off when memory or OS read resources are short, instead of three Pipe workers and several compaction tasks reading TsFiles at once until Windows returns ERROR_NO_SYSTEM_RESOURCES.

What did you see instead?

Pipe workers block 20–39 seconds waiting for parse memory and then report temporary OOM. Concurrently, compaction cannot read a TsFile because the OS reports insufficient system resources. The file involved in the failed read (...\11\2952\...) is not one of the files the Pipe workers were parsing (...\11\2959\... and ...\12\2959\...).

Error

2026-09-24 03:51:20,892 [pool-15-IoTDB-Compaction-Schedule-3] ERROR o.a.t.r.r.LocalTsFileInput:88 - 从当前位置读取 D:\apache-iotdb-2.0.8-all-bin\apache-iotdb-2.0.8-all-bin\data\datanode\data\sequence\root.ShipData\11\2952\1785369603984-1-5-2.tsfile 时出错
2026-09-24 03:51:21,229 [pool-15-IoTDB-Compaction-Schedule-3] ERROR o.a.i.d.s.d.c.s.i.RewriteCrossSpaceCompactionSelector:429 - root.ShipData cannot select file for cross space compaction
org.apache.iotdb.db.exception.MergeException: java.io.IOException: 系统资源不足，无法完成请求的服务。
	at org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl.RewriteCrossSpaceCompactionSelector.selectOneTaskResources(RewriteCrossSpaceCompactionSelector.java:153)
	at org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl.RewriteCrossSpaceCompactionSelector.selectCrossSpaceTask(RewriteCrossSpaceCompactionSelector.java:375)
	...
Caused by: org.apache.tsfile.exception.NotCompatibleTsFileException: java.io.IOException: 系统资源不足，无法完成请求的服务。
	at org.apache.tsfile.read.TsFileSequenceReader.loadFileVersion(TsFileSequenceReader.java:439)
	at org.apache.tsfile.read.TsFileSequenceReader.<init>(TsFileSequenceReader.java:316)
	...
Caused by: java.io.IOException: 系统资源不足，无法完成请求的服务。
	at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at java.base/sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:60)
	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:341)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:307)
	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:284)
	at java.base/sun.nio.ch.FileChannelImpl.implRead(FileChannelImpl.java:251)
	at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:231)
	at org.apache.tsfile.read.reader.LocalTsFileInput.read(LocalTsFileInput.java:83)
	at org.apache.tsfile.read.TsFileSequenceReader.loadFileVersion(TsFileSequenceReader.java:429)

Pipe side, same window:

2026-09-24 03:51:07,196 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-3] INFO  o.a.i.d.p.e.c.t.PipeTsFileInsertionEvent:803 - Wait for memory enough for parsing ...\sequence\root.ShipData\11\2959\1789622109110-279-1-0.tsfile for 20.007 seconds.
2026-09-24 03:51:15,768 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-2] INFO  o.a.i.d.p.e.c.t.PipeTsFileInsertionEvent:803 - Wait for memory enough for parsing ...\sequence\root.ShipData\12\2959\1789624287446-87-0-0.tsfile for 20.02 seconds.
2026-09-24 03:51:19,501 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-2] WARN  o.a.i.c.p.r.l.PipePeriodicalLogReducer:51 - Parse TsFile ...\1789624287446-87-0-0.tsfile error. Because: TimeoutException: Waited 23.753 seconds for memory to parse TsFile
2026-09-24 03:51:25,629 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-1] WARN  ... TimeoutException: Waited 25.243 seconds for memory to parse TsFile (...\1789623847592-339-0-0.tsfile)
2026-09-24 03:51:26,256 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-3] WARN  ... TimeoutException: Waited 39.067 seconds for memory to parse TsFile (...\1789622109110-279-1-0.tsfile)

Anything else?

Timeline (2026-09-24 03:51)

1. Pipe Pool-1, Pool-2, and Pool-3 each log Wait for memory enough for parsing every 10 seconds, on three different sequence TsFiles (root.ShipData partitions 11 and 12). Waits reach 20–30 seconds.
2. Cross-space compaction of root.ShipData-12 starts (1 seq file + 1 unseq file, about 14.7 MB) and finishes.
3. Pipe Pool-2 gives up after 23.753 seconds: TimeoutException: Waited 23.753 seconds for memory to parse TsFile, then Temporarily out of memory in pipe event processing.
4. Unsequence and sequence inner compaction of root.ShipData-11 start together (5 files / 3 MB and 5 files / 9 MB).
5. Cross-space compaction selection for root.ShipData fails while reading sequence\root.ShipData\11\2952\1785369603984-1-5-2.tsfile. Cause: java.io.IOException: 系统资源不足，无法完成请求的服务。
6. The two remaining Pipe workers then time out as well (25.243 s and 39.067 s).
7. Inner compaction of root.ShipData-11 still completes. A JVM compiler replay file replay_pid15624.log was written in this window; hs_err_pid15624.log was not captured.

Are you willing to submit a PR?

• I'm willing to submit a PR!