drivendataorg/cloudpathlib

Methods that move/remove files on the cloud don't remove the cached version

Ouverte

#388 ouverte le 20 déc. 2023

 (3 commentaires) (0 réaction) (0 personne assignée)Python (87 forks)auto 404
bugcachinggood first issue

Métriques du dépôt

Stars
 (624 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

The following code

import cloudpathlib
s3_path = cloudpathlib.S3Path(f"s3://my-bucket/foo.txt")
s3_path.open("x").close() # Create empty file
s3_path.unlink()
s3_path.open("x").close() # Create empty file

does not work as expected. I would expect it to create a file, remove it, and then create it again. Instead, I get

Traceback (most recent call last):
  File "/workspaces/scoutingtool/backend/../cpltest.py", line 9, in <module>
    s3_path.open("x").close() # Create empty file
    ^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/cloudpathlib/cloudpath.py", line 503, in open
    buffer = self._local.open(
             ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/tmp/tmpg5cu85sn/my-bucket/foo.txt'

This seems to happen because the file is kept in cache, even when it was removed from S3. A simple solution would probably to use the "w" mode, even when "x" was supplied.

Guide contributeur