simonw/datasette

Win32 "used by another process" error with datasette publish

Open

#1,479 创建于 2021年9月28日

在 GitHub 查看
 (8 评论) (1 反应) (0 负责人)Python (8,637 star) (625 fork)batch import
bugdatasette-publishhelp wantedwindows

描述

I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

Failed to deploy. Attaching logs:

  1. Tried with an app created via flyctl apps create frosty-fog-8565 and the ran datasette publish fly covid.db --app frosty-fog-8565
Deploying frosty-fog-8565
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080

Error error connecting to docker: An unknown error occured.

Traceback (most recent call last):
  File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
    "--remote-only",
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpgcm8cz66\\frosty-fog-8565'
  1. Tried also with an app that gets autogenerate when running flyctl launch. This also generates the .toml file. Ran then datasette publish fly covid.db --app dark-feather-168 but different error now
==> Validating app configuration

Error not possible to validate configuration: server returned Post "https://api.fly.io/graphql": unexpected EOF

Traceback (most recent call last):
  File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main        
    "__main__", mod_spec)
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
    "--remote-only",
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpnoyewcre\\dark-feather-168'

These are also the contents of the generated .toml file in 2 scenario:

# fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

app = "dark-feather-168"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 6
    timeout = "2s"
  1. But also trying datasette package covid.db to create a local DOCKERFILE to later try to push it via flyctl deploy fails as well.
 => [internal] load build definition from Dockerfile                                                             0.2s 
 => => transferring dockerfile: 396B                                                                             0.0s 
 => [internal] load .dockerignore                                                                                0.1s 
 => => transferring context: 2B                                                                                  0.0s 
 => [internal] load metadata for docker.io/library/python:3.8                                                    4.7s 
 => [auth] library/python:pull token for registry-1.docker.io                                                    0.0s 
 => [internal] load build context                                                                                0.1s 
 => => transferring context: 82.37kB                                                                             0.0s 
 => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3  108.3s 
 => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5  0.0s 
 => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB                   0.0s 
 => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB                21.6s 
 => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB                15.5s 
 => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB                   0.0s 
 => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB                   0.0s 
 => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB                   6.4s 
 => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB                37.7s 
 => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB              82.3s 
 => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB                  26.6s 
 => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366                        5.4s 
 => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB                40.5s 
 => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8                        0.6s 
 => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64                        0.6s 
 => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B                      40.1s 
 => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587                        5.8s 
 => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB                  41.8s 
 => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077                       18.8s 
 => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58                        1.2s 
 => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26                        2.9s 
 => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b                        0.0s 
 => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543                        0.8s 
 => [2/5] COPY . /app                                                                                            2.3s 
 => [3/5] WORKDIR /app                                                                                           0.2s 
 => [4/5] RUN pip install -U datasette                                                                          26.9s 
 => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json                                        3.1s
 => exporting to image                                                                                           1.2s 
 => => exporting layers                                                                                          1.2s 
 => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a                     0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Traceback (most recent call last):
    "__main__", mod_spec)
  File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package
    call(args)
  File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
    tmp.cleanup()
  File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
    _shutil.rmtree(self.name)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpkb27qid3\\datasette'```

贡献者指南

技术栈
pythondocker
领域
backenddevops
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
needs maintainer response
清晰度议题是否清楚说明期望改动、验收标准和下一步。
needs investigation
前置要求
Python basicsDocker installedWindows OS
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
40
研究方向
Investigate the temporary docker directory function in datasette/utils/ init .py (line around 451). The cleanup fails on Windows because the temporary directory is locked by a process (likely Docker). Consider adding a retry mechanism or using a different cleanup strategy (e.g., moving files before removal). Check if Docker Desktop for Windows is running in Windows container mode, which might cause file locks. Also review the fly.io publish plugin's handling of temp directories. See comments in the issue for user's environment details.