Win32 "used by another process" error with datasette publish · simonw/datasette#1479

(8 留言) (1 反應) (0 負責人)Python (8,637 star) (625 fork)batch import

bugdatasette-publishhelp wantedwindows

描述

I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

Failed to deploy. Attaching logs:

Tried with an app created via flyctl apps create frosty-fog-8565 and the ran datasette publish fly covid.db --app frosty-fog-8565

Deploying frosty-fog-8565
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080

Error error connecting to docker: An unknown error occured.

Traceback (most recent call last):
  File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
    "--remote-only",
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpgcm8cz66\\frosty-fog-8565'

Tried also with an app that gets autogenerate when running flyctl launch. This also generates the .toml file. Ran then datasette publish fly covid.db --app dark-feather-168 but different error now

==> Validating app configuration

Error not possible to validate configuration: server returned Post "https://api.fly.io/graphql": unexpected EOF

Traceback (most recent call last):
  File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main        
    "__main__", mod_spec)
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
    "--remote-only",
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpnoyewcre\\dark-feather-168'

These are also the contents of the generated .toml file in 2 scenario:

# fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

app = "dark-feather-168"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 6
    timeout = "2s"

But also trying datasette package covid.db to create a local DOCKERFILE to later try to push it via flyctl deploy fails as well.

 => [internal] load build definition from Dockerfile                                                             0.2s 
 => => transferring dockerfile: 396B                                                                             0.0s 
 => [internal] load .dockerignore                                                                                0.1s 
 => => transferring context: 2B                                                                                  0.0s 
 => [internal] load metadata for docker.io/library/python:3.8                                                    4.7s 
 => [auth] library/python:pull token for registry-1.docker.io                                                    0.0s 
 => [internal] load build context                                                                                0.1s 
 => => transferring context: 82.37kB                                                                             0.0s 
 => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3  108.3s 
 => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5  0.0s 
 => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB                   0.0s 
 => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB                21.6s 
 => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB                15.5s 
 => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB                   0.0s 
 => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB                   0.0s 
 => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB                   6.4s 
 => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB                37.7s 
 => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB              82.3s 
 => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB                  26.6s 
 => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366                        5.4s 
 => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB                40.5s 
 => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8                        0.6s 
 => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64                        0.6s 
 => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B                      40.1s 
 => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587                        5.8s 
 => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB                  41.8s 
 => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077                       18.8s 
 => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58                        1.2s 
 => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26                        2.9s 
 => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b                        0.0s 
 => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543                        0.8s 
 => [2/5] COPY . /app                                                                                            2.3s 
 => [3/5] WORKDIR /app                                                                                           0.2s 
 => [4/5] RUN pip install -U datasette                                                                          26.9s 
 => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json                                        3.1s
 => exporting to image                                                                                           1.2s 
 => => exporting layers                                                                                          1.2s 
 => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a                     0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Traceback (most recent call last):
    "__main__", mod_spec)
  File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package
    call(args)
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpkb27qid3\\datasette'```

貢獻者指南

技術棧: pythondocker
領域: backenddevops
議題類型: bug
難度: 3
預計時間: half day
活動狀態: needs maintainer response
清晰度: needs investigation
前置要求: Python basicsDocker installedWindows OS
新手友善度: 40
研究方向: Investigate the temporary docker directory function in datasette/utils/ init .py (line around 451). The cleanup fails on Windows because the temporary directory is locked by a process (likely Docker). Consider adding a retry mechanism or using a different cleanup strategy (e.g., moving files before removal). Check if Docker Desktop for Windows is running in Windows container mode, which might cause file locks. Also review the fly.io publish plugin's handling of temp directories. See comments in the issue for user's environment details.