Directory listing cache expiry changes when the system clock jumps

Open Beginner friendly
  • Is something wrong?
#2,330 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Dodothereal is already working on this.

Since Oct 29, 2026.

  • #2331 by @IlyaKhalafi — closed without merging
  • #2384 by @UmutAlisoglu — closed without merging
  • #2394 by @Mohamedajab — closed without merging
  • #2398 by @Dodothereal — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
67/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Start in fsspec/dircache.py, where directory listing expiry is implemented, and trace how the configured lifetime is checked during lookup and iteration. Use the controlled-clock reproduction in the issue to verify both forward and backward wall-clock jumps. Done means a fresh listing remains cached after one elapsed second and an old listing expires after six seconds for a five-second lifetime.

Written by the indexing model from the issue text.

Description

Directory listing expiry uses the system clock even though the configured lifetime is an elapsed duration. A forward clock adjustment can expire a fresh listing, and a backward adjustment can keep an old listing alive beyond its lifetime.

Reproduced against master at a46dfd712fbbdb1d032c912ce028a5a1a4b667b0 using controlled clocks without changing the machine clock.

from types import SimpleNamespace
from unittest.mock import patch
from fsspec.dircache import DirCache

for elapsed, wall_jump in [(1, 1000), (6, -1000)]:
    clock = {"wall": 10000, "elapsed": 0}
    fake_time = SimpleNamespace(
        time=lambda: clock["wall"],
        monotonic=lambda: clock["elapsed"],
    )
    with patch("fsspec.dircache.time", fake_time):
        cache = DirCache(listings_expiry_time=5)
        cache["directory"] = []
        clock["wall"] += elapsed + wall_jump
        clock["elapsed"] += elapsed
        print(f"{elapsed=} {wall_jump=} cached={'directory' in cache}")

For the five-second lifetime, the first case should remain cached after one elapsed second, and the second should expire after six. The current results are False and True respectively, rather than True and False. This also affects lookup and iteration.

Dominant language
Python
Stars
1.4k
Forks
570
Avg merge
2d 18h
Merged PRs (30d)
102

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fsspec/filesystem_spec

All issues in fsspec/filesystem_spec

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.