FiberError/NoMemoryError error related to csv gem occurs when upgrading from ruby ​​3.1 to 3.3

Open
#326 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
ruby
Domain
data

Research direction

Start by comparing csv 3.2.5 and 3.2.8, especially commit acc05116c55579b78925dd27024969b9daf69135, and reproduce the reported FiberError, NoMemoryError, and CPU increase under Ruby 3.1.4 or 3.3.5. Done means identifying the regression and adding a fix or regression test showing that the affected CSV loading path no longer fails or causes the reported increase.

Written by the indexing model from the issue text.

Description

When upgrading from ruby 3.1.4 to 3.3.5, FiberError/NoMemoryError error related to csv occurred in our system. In addition, when loading csv file, CPU load also increases compared to before.

  • FiberError: can't alloc machine stack to fiber (1 x 659456 bytes): Cannot allocate memory error
  • NoMemoryError: failed to allocate memory

After investigation we found the cause. It was caused by csv gem. When upgrading ruby ​​version, csv version is also updated (from 3.2.5 to 3.2.8)
Even if we try to keep ruby ​​3.1.4 and only update csv gem to 3.2.8, the FiberError/NoMemoryError error will still occur
We have upgraded ruby ​​3.3.5 and kept the csv gem version 3.2.5 (same as ruby ​​version 3.1.4) and FiberError/NoMemoryError didn't occur and CPU load also didn't increase

It seems that some changes from csv 3.2.5 to 3.2.8 caused the error. This commit of csv gem is very suspicious

Dominant language
Ruby
Stars
220
Forks
139
Avg merge
1d 1h
Merged PRs (30d)
2

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ruby/csv

All issues in ruby/csv

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.