reactor/reactor-core

Reactor core primitives Flux#using, Flux#usingWhen, Flux#fromIterable should be GC friendly to prevent memory leak

Open

#3,199 建立於 2022年9月23日

在 GitHub 查看
 (2 留言) (0 反應) (0 負責人)Java (1,164 fork)batch import
area/performancehelp wantedstatus/need-investigationtype/enhancement

倉庫指標

Star
 (4,748 star)
PR 合併指標
 (平均合併 16天 15小時) (30 天內合併 25 個 PR)

描述

Expected Behavior

I have quite complicated flux "pipe" that uses "large" (~8MB) memory buffers and Iterables which references those buffers. To be more specific I reads avro files from memory using https://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileStream.html and adapts it to Flux with Flux#fromIterable I wrapped creation of DataFileStream in Flux#using which then just calls Flux#fromIterable. I'm expecting resources created with Flux#using to be ready for Garbage Collection right after resourceCleanup is called.

Talking about reactor:

Flux#using and Flux#usingWhen subscriptions should not hold reference to resource and corresponding publisher after resource is cleaned up. Flux#fromIterable subscriptions should not hold reference to iterator after it is exhausted

Actual Behavior

Mentioned primitives holds strong references to resources/iterators preventing them from being garbage collected. If something holds references to subscriptions then resources may not be garbage collected even if they are not required anymore. To overcome this issue I have to create some strange wrappers which wraps Resources, Iterable and Iterators.

Steps to Reproduce

The minimal reproduceable example I can create: https://gist.github.com/alopukhov/50a8d2e953346692e2c4c819c7c7d065 Please limit heap to 128 MB. It doesn't produce OOM only with specific resource wrapper (testWrapper2).

Possible Solution

Nullify references to resources after they are not required anymore.

Your Environment

  • Reactor version(s) used: reactor-core 3.4.23, 3.4.7

貢獻者指南