`bst workspace open` dislikes git sources that have submodules

Start with buildstream/plugins/sources/git.py at init_workspace(), then trace the URL handling through source.py's translate_url() and mark_download_url(). Reproduce with bst workspace open desktop/gst-libav gst-libav using a Git source with submodules, and verify that the workspace opens without the "Primary URL marked twice" assertion.

See original issue on GitLab
In GitLab by [Gitlab user @coldtom] on Nov 8, 2018, 10:41

Summary

Attempting to run bst workspace open caused me a stack trace, I think due to the submodules in a git element, but this may not be the case.

Steps to reproduce

1. Clone freedesktop-sdk
2. Run bst workspace open desktop/gst-libav gst-libav

What is the current bug behavior?

The following stack trace:

[--:--:--][][] START   Loading elements
[00:00:01][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:01][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources
[--:--:--][d4ca660b][ main:desktop/gstreamer-libav.bst   ] START   Staging sources to here
[--:--:--][        ][ main:desktop/gstreamer-libav.bst-0 ] STATUS  Running host command

    /usr/bin/git show 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2:.gitmodules

[--:--:--][        ][ main:desktop/gstreamer-libav.bst-0 ] STATUS  Running host command

    /usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 common

[--:--:--][        ][ main:desktop/gstreamer-libav.bst-0 ] STATUS  Running host command

    /usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 gst-libs/ext/libav

[--:--:--][        ][ main:desktop/gstreamer-libav.bst-0 ] STATUS  Running host command

    /usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 gst-libs/ext/gas-preprocessor

[--:--:--][        ][ main:desktop/gstreamer-libav.bst-0 ] START   Setting up workspace "/home/tom/.cache/buildstream/build/workspace-desktop-gstreamer-libavyrmfg9lh"
[--:--:--][][] BUG     Primary URL marked twice with different URLs

    Traceback (most recent call last):
      File "/home/tom/.local/bin/bst", line 8, in <module>
        sys.exit(cli())
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
        return self.main(*args, **kwargs)
      File "/home/tom/Documents/work/buildstream/buildstream/_frontend/cli.py", line 162, in override_main
        standalone_mode=standalone_mode, **extra)
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 697, in main
        rv = self.invoke(ctx)
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "/home/tom/.local/lib/python3.7/site-packages/click/decorators.py", line 27, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/home/tom/Documents/work/buildstream/buildstream/_frontend/cli.py", line 705, in workspace_open
        force=force)
      File "/home/tom/Documents/work/buildstream/buildstream/_stream.py", line 513, in workspace_open
        target._open_workspace()
      File "/home/tom/Documents/work/buildstream/buildstream/element.py", line 1810, in _open_workspace
        source._init_workspace(temp)
      File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 672, in _init_workspace
        self.init_workspace(directory)
      File "/home/tom/Documents/work/buildstream/buildstream/plugins/sources/git.py", line 408, in init_workspace
        mirror.init_workspace(directory)
      File "/home/tom/Documents/work/buildstream/buildstream/plugins/sources/git.py", line 220, in init_workspace
        url = self.source.translate_url(self.url)
      File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 455, in translate_url
        self.mark_download_url(url, primary=primary)
      File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 502, in mark_download_url
        "Primary URL marked twice with different URLs"
    AssertionError: Primary URL marked twice with different URLs

What is the expected correct behavior?

Workspace should open correctly.

• BuildStream version affected: /milestone %BuildStream_v1.2