dvc exp run: fails when used with github actions and CML docker container

Riproduci il malfunzionamento nel workflow GitHub Actions stale-model-example usando il container Docker CML e confrontalo con l’esecuzione SSH interattiva riuscita. Inizia da dvc/commands/experiments/run.py e dvc/repo/experiments/queue/workspace.py, quindi esamina il malfunzionamento scmrepo/pygit2 segnalato. Il lavoro è completato quando dvc get scarica i dati richiesti senza l’errore di repository non trovato.

Description

dvc get fails in one of our example repos with the error response:

ERROR: unexpected error - Repository not found at /__w/stale-model-example/stale-model-example/.git/

None of the directories in that path are symlinks.

Running the dvc get command interactively (in an ssh session via action-tmate results in success - the action completes without error.

Running with the -vv flag gives the following output:

2022-09-28 18:08:49,331 TRACE: Namespace(all_pipelines=False, cd='.', checkpoint_resume=None, cmd='run', cprofile=False, cprofile_dump=None, downstream=False, dry=False, force=False, force_downstream=False, func=<class 'dvc.commands.exper
iments.run.CmdExperimentsRun'>, instrument=False, instrument_open=False, interactive=False, jobs=1, machine=None, metrics=False, name=None, parser=DvcParser(prog='dvc', usage=None, description='Data Version Control', formatter_class=<clas
s 'argparse.RawTextHelpFormatter'>, conflict_handler='error', add_help=False), pdb=False, pipeline=False, pull=False, queue=False, quiet=0, recursive=False, reset=False, run_all=False, set_param=[], single_item=False, targets=[], tmp_dir=
False, verbose=2, version=None, viztracer=False, viztracer_depth=None, yappi=False)                                                                                                                                                           
2022-09-28 18:08:49,709 ERROR: unexpected error - Repository not found at /__w/stale-model-example/stale-model-example/.git/
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dvc/cli/__init__.py", line 185, in main
    ret = cmd.do_run()
  File "/usr/local/lib/python3.8/dist-packages/dvc/cli/command.py", line 22, in do_run
    return self.run()
  File "/usr/local/lib/python3.8/dist-packages/dvc/commands/experiments/run.py", line 32, in run
    results = self.repo.experiments.run(
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/__init__.py", line 521, in run
    return run(self.repo, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/run.py", line 60, in run
    return repo.experiments.reproduce_one(
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/__init__.py", line 119, in reproduce_one
    self.queue_one(exp_queue, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/__init__.py", line 156, in queue_one
    return self.new(
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/utils.py", line 40, in wrapper
    return f(exp, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/__init__.py", line 269, in new
    return queue.put(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/queue/workspace.py", line 25, in put
    return self._stash_exp(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/dvc/repo/experiments/queue/base.py", line 315, in _stash_exp
    with self.scm.detach_head(client="dvc") as orig_head:
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/scmrepo/git/__init__.py", line 404, in detach_head
    self.checkout(rev, detach=True, force=force)
  File "/usr/local/lib/python3.8/dist-packages/scmrepo/git/__init__.py", line 283, in _backend_func
    for key, backend in self.backends.items():
  File "/usr/lib/python3.8/_collections_abc.py", line 744, in __iter__
    yield (key, self._mapping[key])
  File "/usr/local/lib/python3.8/dist-packages/scmrepo/git/__init__.py", line 49, in __getitem__
    initialized = backend(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/scmrepo/git/backend/pygit2.py", line 97, in __init__
    self.repo = pygit2.Repository(path)
  File "/usr/local/lib/python3.8/dist-packages/pygit2/repository.py", line 1620, in __init__
    path_backend = init_file_backend(path, flags)
_pygit2.GitError: Repository not found at /__w/stale-model-example/stale-model-example/.git/
------------------------------------------------------------
2022-09-28 18:08:50,051 DEBUG: Version info for developers:
DVC version: 2.28.0 (pip)
---------------------------------
Platform: Python 3.8.10 on Linux-5.15.0-1020-azure-x86_64-with-glibc2.29
Subprojects:
        dvc_data = 0.13.0
        dvc_objects = 0.5.0
        dvc_render = 0.0.11
        dvc_task = 0.1.2
        dvclive = 0.11.0
        scmrepo = 0.1.1
Supports:
        gs (gcsfs = 2022.8.2),
        http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        s3 (s3fs = 2022.8.2, boto3 = 1.24.59)
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: s3, gs
Workspace directory: ext4 on /dev/root
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-09-28 18:08:50,053 DEBUG: Analytics is enabled.
2022-09-28 18:08:50,117 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp3gfjynwi']'
2022-09-28 18:08:50,119 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp3gfjynwi']'

Reproduce

The only way of reproducing this that I've found is via github actions.

Expected

The dvc get command should download the requested data.

Environment information

Output of dvc doctor:

DVC version: 2.30.0 (pip)
---------------------------------
Platform: Python 3.8.10 on Linux-5.15.0-1020-azure-x86_64-with-glibc2.29
Subprojects:
	dvc_data = 0.17.1
	dvc_objects = 0.7.0
	dvc_render = 0.0.12
	dvc_task = 0.1.3
	dvclive = 0.12.0
	scmrepo = 0.1.1
Supports:
	gs (gcsfs = 2022.8.2),
	http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
	s3 (s3fs = 2022.8.2, boto3 = 1.24.59)
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: s3, gs
Workspace directory: ext4 on /dev/root
Repo: dvc, git