FTP reads inside a transaction use a nonexistent staging path

Open Beginner friendly
  • Is something wrong?
#2,463 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Harsh23Kashyap is already working on this.

Since Nov 15, 2026.

  • #2464 by @Harsh23Kashyap — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Start in fsspec/implementations/ftp.py, at FTPFile.__init__, where the requested path is swapped for a random path under tempdir whenever autocommit=False; read AbstractFileSystem.open and the transaction context in fsspec/spec.py to see how autocommit reaches the file. The guard should apply only to write modes so rb keeps the caller's path. Verify with the local pyftpdlib setup in fsspec/implementations/tests/test_ftp.py, adding cases for text/binary reads, explicit autocommit=False, reader lifetime, read-then-write rollback/commit, and rerun the existing write-transaction tests. Note the reporter says they are already preparing this fix, so check the thread before starting.

Written by the indexing model from the issue text.

Description

Problem
  • Reading a committed FTP file inside fs.transaction fails with ftplib.error_perm: 550 No such file or directory. The same read outside the transaction succeeds. Reproduced twice against a local pyftpdlib 2.2.0 server on master 12c823bc.
  • FTPFile.__init__ replaces the requested path with a random staging path whenever autocommit=False, including rb. This prevents read-then-write transactions from reading their input.
# The FTP server already contains /input and an empty /stage directory.
fs = FTPFileSystem(host, port, username, password, tempdir="/stage")
with fs.open("/input", "rb") as f:
    assert f.read() == b"committed input"
with fs.transaction:
    with fs.open("/input", "rb") as f:
        f.read()  # 550: attempts RETR /stage/<random UUID>
Proposed scope
  • Keep read-only opens on the requested path, including explicit autocommit=False. Continue staging writes. No API or data-model change, migration, or transactional append redesign.
  • Regression checks should cover binary/text reads, explicit autocommit settings, reader lifetime, and read-then-write commit/rollback without changing the input. Existing write transaction tests must still pass.
  • Changing AbstractFileSystem.open globally would affect unrelated backends; a local FTP guard avoids that. The main risk is changing writer staging, so those controls are part of the fix.
  • The analogous SFTP change in #2462 touches a different backend. The FTP append work in #2403 is separate. I am preparing the narrow FTP fix and local-server tests.
Dominant language
Python
Stars
1.4k
Forks
577
Avg merge
2d 36m
Merged PRs (30d)
51

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fsspec/filesystem_spec

All issues in fsspec/filesystem_spec

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.