drivendataorg/cloudpathlib
Methods that move/remove files on the cloud don't remove the cached version
Offen
#388 geöffnet am 20.12.2023
bugcachinggood first issue
Repository-Metriken
- Stars
- (624 Sterne)
- PR-Merge-Metriken
- (PR-Metriken ausstehend)
Beschreibung
The following code
import cloudpathlib
s3_path = cloudpathlib.S3Path(f"s3://my-bucket/foo.txt")
s3_path.open("x").close() # Create empty file
s3_path.unlink()
s3_path.open("x").close() # Create empty file
does not work as expected. I would expect it to create a file, remove it, and then create it again. Instead, I get
Traceback (most recent call last):
File "/workspaces/scoutingtool/backend/../cpltest.py", line 9, in <module>
s3_path.open("x").close() # Create empty file
^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/cloudpathlib/cloudpath.py", line 503, in open
buffer = self._local.open(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/pathlib.py", line 1044, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/tmp/tmpg5cu85sn/my-bucket/foo.txt'
This seems to happen because the file is kept in cache, even when it was removed from S3. A simple solution would probably to use the "w" mode, even when "x" was supplied.