Failed to call _flushSendQueue() during frequent log emit

Open
#168 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
32/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
javascript, node.js

Research direction

Start by tracing _connect(), the tls.connect callback, and _flushSendQueue() to understand how the connecting state and queued logs interact. Reproduce the frequent emit scenario described in the issue, then determine the expected behavior for queued data and memory use while the socket is connecting.

Written by the indexing model from the issue text.

Description

Hi,
We're using fluent-logger-node v3.4.1 to send logs from Lambda functions. During our test, we emit 300KB logs every 100ms for 60 seconds, so around 180MB logs in total. During the test, the first emit() started a socket connection(see code below)
if (this.tls) { this._socket = tls.connect(Object.assign({}, this.tlsOptions, { host: this.host, port: this.port }), () => { postConnect(); }); addHandlers(); }
Somehow the call back function was not called until the last emit. For the following emit(), the _connect() returned immediately because it thought still in a connecting process. So all logs were pushed into the queue and flushed once after the last emit(). I wonder if this is the expected behavior because we may run out of memory if log size is big.

Dominant language
JavaScript
Stars
258
Forks
82
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fluent/fluent-logger-node

All issues in fluent/fluent-logger-node

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.