[BUG] FileStore::Directory#list returns nothing when a file vanishes mid-listing
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 32/100
Research direction
Start in the file store's directory implementation, at FileStore::Directory#list and #walk, where Find.find yields names and File.lstat raises. The reporter's stub deletes one file right before lstat, so reproduce with that before changing anything. Done means a listing with a vanished name still returns every other file, while permission errors still end the listing — but note an open PR (#1676) already proposes exactly this fix, so coordinate before writing code.
Written by the indexing model from the issue text.
Description
Basic checks
- I searched existing issues - this hasn't been reported
- I can reproduce this consistently
- This is a RubyLLM bug, not my application code
What's broken?
FileStore::Directory#list walks the folder with Find.find, which reads a folder's names and then yields them one at a time. #walk calls File.lstat on each name it is given. When a name disappears between the two, Errno::ENOENT escapes #walk, and the rescue SystemCallError in #list turns the whole listing into []. The same happens with Errno::ENOTDIR when a folder is replaced by a file.
The store causes this itself: #write writes a hidden temporary file and renames it into place. #list does not take the store's lock, so a listing that runs while another thread or process writes can see the temporary name and lose it before lstat. list_files then tells the model the chat has no files, and search_files finds nothing. Find already skips names that vanish when it checks them itself, so the file store is the only place that turns one vanished name into an empty result.
This is not about which files a listing shows while it races a write. It is about losing every other file in the folder.
How to reproduce
require "ruby_llm"
require "tmpdir"
Dir.mktmpdir do |folder|
store = RubyLLM::FileStore::Directory.new(folder)
store.write("a.txt", "a")
writer = Thread.new { 3000.times { |i| store.write("notes-#{i % 5}.txt", i.to_s) } }
listings = []
listings << store.list.size while writer.alive?
writer.join
puts "#{listings.count(0)} of #{listings.size} listings were empty"
end
a.txt exists the whole time, so no listing should be empty. On my machine 279 to 519 of every 6,000 to 8,000 listings came back empty (about 4 to 7 percent). The rate depends on timing, but the cause is deterministic: a stub that deletes one file right before File.lstat reaches it empties the listing every time.
Expected behavior
list returns the files that are still there, and leaves out only the one that vanished.
What actually happened
519 of 7884 listings were empty
I have a small local fix with specs and can open a PR: #walk skips a name that raises Errno::ENOENT or Errno::ENOTDIR, as Find does. Other errors, such as a permission error, still end the listing as they do now.
Environment
- Ruby version: ruby 3.3.5
- RubyLLM version:
mainat9c20ca61 - Provider: none (file store only)
- OS: macOS (Darwin 27, APFS)
- Dominant language
- Ruby
- Stars
- 4.5k
- Forks
- 537
- Avg merge
- 7h 30m
- Merged PRs (30d)
- 378
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from crmne/ruby_llm
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 55/100
crmne/ruby_llm#1682 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 22/100
crmne/ruby_llm#1674 · 1 comment ·
Maintainers usually reply within 1 day
-
[BUG] FileStore::Directory follows links to folders inside its own folderPossibly taken @yorzi claimed this 1 day ago. Openbug
Difficulty 3/5 1-2 days Newbie friendliness 30/100
crmne/ruby_llm#1673 · 1 comment ·
Maintainers usually reply within 1 day
-
Body logging is tied to the logger's debug level, and streaming debug lines ignore the context loggerPossibly taken @josh-sanders claimed this 1 day ago. Openbug
Difficulty 3/5 Half a day Newbie friendliness 55/100
crmne/ruby_llm#1669 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Bug Ruby
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
getsentry/sentry-ruby#3211 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
internetee/registry#2984 ·
Maintainers usually reply within 6 days
-
interest-zsv
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
hitobito/hitobito_zsv#6 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ecosyste-ms/issues#862 ·
Maintainers usually reply within 1 day
-
Redis Lua error `@user_script:52: ERR value is not an integer or out of range` when `window_size` is configuredPossibly taken @mimosa claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day