Project.validate() can't detect content changes served by a custom fscommands

Open
  • Is something wrong?
#925 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 7 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
devtools

Research direction

Start from the reproduction script in the issue, which shows that Project.validate() keeps the cached module after an fscommands read() override changes the content. Read ChangeIndicator, which compares os.path.getmtime() and os.path.getsize(), and the Project.validate() path that uses it. The issue offers two designs (a forced invalidation for one resource, or an optional change-indicator hook on fscommands), so the maintainers need to choose one before a patch. Done means the cached module reflects the overlay text after validate(), with existing fscommands implementations still working.

Written by the indexing model from the issue text.

Description

A custom fscommands.read() lets an editor or a language server give Rope the contents of unsaved buffers. pylsp-rope's WorkspaceFileCommands does this, and so will rope-language-server. Rope caches parsed modules, though, and Project.validate() decides whether a file changed only from ChangeIndicator, which calls os.path.getmtime() and os.path.getsize() on the real file. An edit to a buffer changes neither, so Rope keeps analysing the old module even after validate().

To Reproduce

import pathlib
import tempfile

from rope.base import fscommands
from rope.base.project import Project


class Overlay(fscommands.FileSystemCommands):
    """Serve unsaved editor buffers, fall back to the disk."""

    def __init__(self):
        self.buffers = {}

    def read(self, path):
        if path in self.buffers:
            return self.buffers[path].encode()
        return super().read(path)


root = pathlib.Path(tempfile.mkdtemp())
(root / "m.py").write_text("def f():\n    return 1\n")
overlay = Overlay()
project = Project(str(root), ropefolder=None, fscommands=overlay)
resource = project.get_resource("m.py")
print(sorted(project.get_pymodule(resource).get_attributes()))

overlay.buffers[resource.real_path] = "def g():\n    return 2\n"
project.validate(resource)
print(sorted(project.get_pymodule(resource).get_attributes()))
print(resource.read())

Output on master (46ebf37), Python 3.13:

['f']
['f']
def g():
    return 2

resource.read() returns the new text, but the cached module still has f and not g. Refactorings then compute offsets against text the user no longer has.

Workarounds today

  • pylsp-rope calls the private project.pycore._invalidate_resource_cache(resource) for every open document on every request.
  • libutils.report_change() does invalidate, but the documentation describes it as the call to make after saving, and with the default automatic_soa=True it also runs static object analysis on every call. That's too slow to run on every keystroke.

Proposal

Either of these would be enough, and neither changes the existing fscommands contract:

  1. A public way to invalidate one resource, for example project.invalidate(resource) or project.validate(resource, force=True). This only needs to notify the resource observers, without the analysis that report_change() adds.
  2. An optional hook, for example fscommands.get_change_indicator(path), which ChangeIndicator uses when the fscommands object provides it, and falls back to os.stat() otherwise. An overlay could then return the buffer version.

The second option is close to part of #911, but kept optional, which matches the concern raised in the review of #911 that new fscommands methods must not break existing implementations such as pylsp-rope's.

Editor information

Dominant language
Python
Stars
2.2k
Forks
197
Avg merge
2d 13h
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python-rope/rope

All issues in python-rope/rope

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.