Request for an alternative to supporting dynamic imports - need a way to pass a reference path (library path?)

Start with the dynamic-import examples and the Python jsonnet.evaluate_file entry point described in the issue, then trace how imports are resolved and where the computed-import restriction is enforced. The issue names no repository files or tests; done means a supported way to provide a changing base or library path for imports, while preserving existing relative imports and calls that do not use it.

Hello all, and sorry for resurrecting the dynamic include topic again. I appreciate this has been answered numerous times, and i see there is at least one way to approximate dynamic includes, but this doesn't effectively solve the issue i have at hand.

I'm an extensive user of Jsonnet, i'm using it to generate very complex configurations via many levels of "import"s, allowing me to have extremely short top-level .jsonnet files, promoting great configuration code reuse. For reference, i'm rendering the Jsonnet in python, and using the latest v0.20.0 version.

The problem i'm facing is in using jsonnet together with jenkins build pipelines, where the job folder path is dynamic, and i'm using two separate separate repositories in this single pipeline (and said repositories absolutely must stay separate). To allow parallelism on jenkins (building the same pipeline with different revisions), we have adopted a "job folder" naming, which i call the base path, as /.../<job_name>/<job_id>/. This is unique per every job.

I'm using jsonnet in both repositories, lets call them A at /.../<job_name>/<job_id>/A and B at /.../<job_name>/<job_id>/B. at The B repository has the top level .jsonnet files alongside many other .jsonnets that get imported via relative paths, so this is all good. However, i additionally need to import some configuration files from A, for example, /.../<job_name>/<job_id>/A/cfg1.jsonnet, and cfg2, cfg3 - cfgN . It's obvious that this import name isnt something that's needs to be created "dynamically", e.g. the file name and path relative to the base path is fixed, but the base path changes with every run, and there is seemingly no way to do local cfg1 = import ref_path + "cfg1.jsonnet".

I've considered numerous solutions to this problem:

• local relative_path = "/.../<job_name>/<job_id>/A/"; local cfg1 = import relative_path + "cfg1.jsonnet" - this is the ideal solution, obviously doesn't work due to jsonnet computed imports are not allowed
• Hardcoding a relative path from B to A like ../A/cfg1.jsonnet - not possible in my case, because the name A changes between different types of pipelines, and therefore it cannot be hardcoded.
• Importing configs via ext_codes, e.g. ext_codes = { cfg1 : "/.../<job_name>/<job_id>/A/cfg1.jsonnet, cfg2:..., cfg3:..."} - not possible either because i may have two sets of A repositories in use, e.g. /.../<job_name>/<job_id>/A1, /.../<job_name>/<job_id>/A/, and additionally, it forces passing these arguments every time, but there are cases where the top level .jsonnet doesn't include anything from A at all (meaning i would have to have at minimum two different calls to the jsonnet.evaluate_file, AND it would require me to know if A is in use or not, further complicating the jsonnet rendering process)
• handling the entirety of A configs separately from B and merging them together in the backend - this is what is currently done, and i'm writing this post specifically for help to move away as its not flexible enough for my needs. I want to be able to dynamically modify and use the configurations from A inside B, e.g. take one parameter from cfg1 and sweep it, creating multiple copies of cfg1, in a .jsonnet file in B

I also have an additional complexity of there being a A/reshaper.jsonnet with a function in it that is to be included in .jsonnet files from B, and will take the config data from A.

For a more realistic example of what i'm trying to do, here is an extremely short version of what i'm trying to accomplish.

• /common_path/A1/cfg1.jsonnet

{
    name: "cfg1",
    size: 1
}

• /common_path/A1/reshaper.jsonnet

local reshape(data) = [
   "--" + idx.key + "=" + idx.value
   for idx in std.objectKeysValues(data)
]
{
    reshape: reshape
}

• similarly, duplicate the above for /comon_path/A2/...

• /common_path/B/top_X.jsonnet

local cfg1 = import std.extVal("common_path") + "A1/cfg1.jsonnet"
local cfg2 = import std.extVal("common_path") + "A1/cfg2.jsonnet"
local cfgN = import std.extVal("common_path") + "A1/cfgN.jsonnet"
local reshaper = import std.extVal("common_path") + "A1/reshaper"
local reshape = reshaper.reshape

local cfg1_modified = ...
local cfg2_modified = ...
local cfgN_modified = ...

{
   "banana" : 1,
   "apple" : 2,
   "cfgs" : [reshape(cfg1), reshape(cfg2)...reshape(cfgN)]
}

• /common_path/B/top_Y.jsonnet

local cfg1 = import std.extVal("common_path") + "A2/cfg1.jsonnet"
local reshaper = import std.extVal("common_path") + "A2/reshaper.jsonnet"
<similar to top_X.jsonnet>
{}

• /common_path/B/top_Z.jsonnet

<doesn't include anything from A1 or A2>
{}

• render as follows:
  • jsonnet.evaluate_file("<..>/top_X.jsonnet", ext_vars="{common_path : "/common_path/}") - uses the common_path
  • jsonnet.evaluate_file("<..>/top_Y.jsonnet", ext_vars="{common_path : "/common_path/}") - uses the common_path
  • jsonnet.evaluate_file("<..>/top_Z.jsonnet", ext_vars="{common_path : "/common_path/}") - ignores the common_path but still works fine

Before you say it... yes, i am doing something horribly complex, and thankfully jsonnet so far has been able to cope with absolutely everything i throw at it so far, except for this current common path problem i'm trying to solve. (To give you a sense of scale here, one of the top_X.jsonnet files in my use cases is 51 lines long, and renders into json that's >3000 lines long).

I hope i'm just missing something very obvious and simple here, something like an way to pass a library path and import files relative to this library path.