The firebase-admin-java 9.4.0 does not work with NIO (Non-blocking I/O).

Open
#1,029 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
java
Domain
api, backend

Research direction

Start by tracing FirebaseMessaging.sendAsync through the FirebaseOptions ThreadManager configuration and FirebaseThreadManagers.DefaultThreadManager.doInit(). Compare the firebase thread lifecycle with the httpclient-dispatch and IO selector behavior described in the interceptors. The issue does not specify a file, test, or accepted resolution, so confirm the intended handoff and reproducible failure before defining what done means.

Written by the indexing model from the issue text.

Description

api: core type: feature request
[READ] Step 1: Are you in the right place?
  • For issues or feature requests related to the code in this repository
    file a Github issue.
    • If this is a feature request make sure the issue title starts with "FR:".
  • For general technical questions, post a question on StackOverflow
    with the firebase tag.
  • For general Firebase discussion, use the firebase-talk
    google group.
  • For help troubleshooting your application that does not fall under one
    of the above categories, reach out to the personalized
    Firebase support channel.
[REQUIRED] Step 2: Describe your environment
  • Operating System version: ubuntu
  • Firebase SDK version: firebase-admin-java 9.4.0
  • Library version: java 17
  • Firebase Product: fcm
[REQUIRED] Step 3: Describe the problem

Hello,
I am testing the firebase-admin-java SDK 9.4.0 with the following configuration:

FirebaseOptions options = FirebaseOptions.builder()
    .setCredentials(googleCredentials(firebaseCredential, httpTransport))
    .setDatabaseUrl(firebaseDatabaseUrl)
    .setHttpTransport(httpTransport)
    .setConnectTimeout(FIREBASE_CONNECT_TIMEOUT)
    .setReadTimeout(FIREBASE_READ_TIMEOUT)
    .setThreadManager(new CustomFirebaseThreadManager(
        firebaseTaskExecutor.getThreadPoolExecutor()))
    .build();

The reason I explicitly added ThreadManager is to monitor the ThreadPoolExecutor. Without this configuration, the ThreadPoolExecutor generated by FirebaseThreadManagers.DefaultThreadManager.doInit() will be used, which has a queue size of Integer.MAX_VALUE. If FCM responses are delayed, the queue can pile up, potentially causing an OOM (OutOfMemoryError).

Below is the configuration for H2AsyncClientBuilder:

H2AsyncClientBuilder h2AsyncClientBuilder = H2AsyncClientBuilder.create()
    .setDefaultConnectionConfig(connectionConfig)
    .evictIdleConnections(TimeValue.of(Duration.ofMinutes(10)))
    .setDefaultRequestConfig(requestConfig())
    .setIOReactorConfig(IOReactorConfig.custom()
        .setSoTimeout(FIREBASE_READ_TIMEOUT, TimeUnit.MILLISECONDS)
        .build())
    .addRequestInterceptorFirst(
        (HttpRequest request, EntityDetails entity, HttpContext context) -> {
            activeStream.incrementAndGet();
            if (context != null) {
                context.setAttribute("startTime", System.nanoTime());
            }
        })
    .addResponseInterceptorFirst(
        (HttpResponse response, EntityDetails entity, HttpContext context) -> {
            if (context == null || context.getAttribute("startTime") == null) {
                return;
            }
            long startTime = Long.parseLong(context.getAttribute("startTime").toString());
            long endTime = System.nanoTime();
            if (startTime < endTime) {
                Timer.builder(FCM_RESPONSE_TIMER_NAME)
                    .publishPercentiles(0.50, 0.95, 0.99)
                    .tags(
                        Lists.newArrayList(
                            Tag.of("host", ((HttpRoute) context.getAttribute("http.route")).getTargetHost().getHostName()),
                            Tag.of("http_version", context.getProtocolVersion().toString())
                        )
                    )
                    .register(meterRegistry)
                    .record(endTime - startTime, TimeUnit.NANOSECONDS);
            }
            activeStream.decrementAndGet();
        })
    .disableCookieManagement()
    .setH2Config(H2Config.custom().setMaxConcurrentStreams(500 * 10).build())
    .disableRedirectHandling()
    .disableAutomaticRetries();

Here is the code for sending FCM messages. The postProcessExecutor is simply responsible for logging the send results:

ApiFuture<String> responseApiFuture = firebaseMessaging.sendAsync(fcmMessage);
responseApiFuture.addListener(() -> {
    try {
        responseApiFuture.get();
        message.setResponseCode(FcmResult.SUCCESS.getCode());
        postProcessor.postProcess(MessageStatus.SENT, message);
    } catch (Exception e) {
        log ~~~
    }
}, postProcessorExecutor);

When testing this, if I set a breakpoint inside addRequestInterceptorFirst (or addResponseInterceptorFirst) in the second code block, the thread executing addRequestInterceptorFirst is the httpclient-dispatch thread.
At this point, firebaseThreadPoolTask.getActiveCount() (as set in FirebaseOptions.builder().setThreadManager()) is 1.
In other words, it seems like the firebaseThread is staying active while the httpclient-dispatch thread is running.

I would like the behavior to work as follows:

The firebaseThread performs tasks (such as SDK validation) before the HTTP call.
After handing over to httpclient-dispatch, the firebaseThread is released.
The httpclient-dispatch makes the request to the FCM server, and the IO selector detects when a response is received.
Once the IO selector detects the response, it hands over to httpclient-dispatch, which processes operations like interceptors and then passes it to postProcessorExecutor.
This behavior should prevent the firebaseThread from getting blocked when the FCM server is down or latency is high.
If the firebaseThread gets blocked, it means that the queue in the thread pool fills up, which can lead to OOM or RejectedExecutionException.

If I have misunderstood anything, please let me know.
If my understanding is correct, how can I resolve this issue?

Dominant language
Java
Stars
620
Forks
305
Avg merge
3h 23m
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from firebase/firebase-admin-java

All issues in firebase/firebase-admin-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.