reactor/reactor-core

Reactor core primitives Flux#using, Flux#usingWhen, Flux#fromIterable should be GC friendly to prevent memory leak

Open

#3,199 opened on 2022年9月23日

GitHub で見る
 (2 comments) (0 reactions) (0 assignees)Java (1,164 forks)batch import
area/performancehelp wantedstatus/need-investigationtype/enhancement

Repository metrics

Stars
 (4,748 stars)
PR merge metrics
 (平均マージ 16d 15h) (30d で 25 merged PRs)

説明

Expected Behavior

I have quite complicated flux "pipe" that uses "large" (~8MB) memory buffers and Iterables which references those buffers. To be more specific I reads avro files from memory using https://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileStream.html and adapts it to Flux with Flux#fromIterable I wrapped creation of DataFileStream in Flux#using which then just calls Flux#fromIterable. I'm expecting resources created with Flux#using to be ready for Garbage Collection right after resourceCleanup is called.

Talking about reactor:

Flux#using and Flux#usingWhen subscriptions should not hold reference to resource and corresponding publisher after resource is cleaned up. Flux#fromIterable subscriptions should not hold reference to iterator after it is exhausted

Actual Behavior

Mentioned primitives holds strong references to resources/iterators preventing them from being garbage collected. If something holds references to subscriptions then resources may not be garbage collected even if they are not required anymore. To overcome this issue I have to create some strange wrappers which wraps Resources, Iterable and Iterators.

Steps to Reproduce

The minimal reproduceable example I can create: https://gist.github.com/alopukhov/50a8d2e953346692e2c4c819c7c7d065 Please limit heap to 128 MB. It doesn't produce OOM only with specific resource wrapper (testWrapper2).

Possible Solution

Nullify references to resources after they are not required anymore.

Your Environment

  • Reactor version(s) used: reactor-core 3.4.23, 3.4.7

コントリビューターガイド