[EngSys] Detect compiled dev requirements deterministically instead of regex-matching [tool.cibuildwheel]

The natural instinct is to reuse what the build pipeline does. steps/resolve-build-platforms.yml sets ENABLE_EXTENSION_BUILD by reading the PackageInfo JSON folder.

Two independent blockers:

1. The folder doesn't exist in this job. live.tests.yml calls resolve-package-targeting.yml without PackagePropertiesFolder, which defaults to ''. No PackageInfo folder is produced anywhere in the live test chain. jobs/ci.tests.yml does pass it; jobs/live.tests.yml does not.

2. The schema has no field for this anyway. eng/scripts/get_package_properties.py emits only name, version, is_new_sdk, folder, and dependent_packages. There is nothing about extensions or compilation.

That is precisely why resolve-build-platforms.yml resorts to:

if ($packageProperties -contains "azure-storage-extensions") {

Adopting this pattern would trade a regex for a hardcoded package name — less deterministic, not more. Worth noting the build pipeline has the same latent problem, just expressed differently.

A variant would be to add an ext_modules / is_compiled field to the package properties schema and plumb PackagePropertiesFolder through live.tests.yml. That's a larger change touching cross-language tooling (Package-Properties.ps1 consumers), and it makes the live test job depend on a save-package-properties step it doesn't currently run.

Keep the current shape and scope; just fix the predicate. Ask the same parser create_package uses:

from ci_tools.parsing import ParsedSetup
ParsedSetup.from_path(package_dir).ext_modules

Walk the package directories under the service directory and check .ext_modules instead of regex-matching the file.

Pros: roughly a one-line semantic change; removes the proxy entirely; no parameter plumbing.

Cons: still service-directory-scoped rather than target-scoped; introduces a Python dependency into a step that is currently pure PowerShell (see the ordering constraint below).

Replicate what actually happens at runtime. For each package in $(TargetingString), read its dev_requirements.txt, filter with is_relative_install_path, and check .ext_modules on the resolved target:

for pkg in discover_targeted_packages(targeting_string, os.path.join("sdk", service_dir)):
    for req in open(os.path.join(pkg, "dev_requirements.txt")):
        if is_relative_install_path(req.strip(), pkg):
            if ParsedSetup.from_path(os.path.abspath(os.path.join(pkg, req.strip()))).ext_modules:
                # cibuildwheel will run

This is a line-for-line mirror of build_whl_for_req — the function that invokes cibuildwheel. It uses the same ParsedSetup, the same is_relative_install_path, and the same dev-requirements traversal, so it cannot drift from the behavior it is predicting.

I prototyped this locally against the real repo:

It also tightens the gate from service directory to packages actually under test, addressing the secondary issue above.

Pros: correct predicate, correct scope, structurally incapable of diverging from runtime behavior.

Cons: most code; same ordering constraint as B1; ~1s per package parse cost (ParsedSetup execs setup.py).

ci_tools is not importable where the step currently sits. In live.tests.yml the new steps run before the build-test.yml template, and everything needed arrives inside it:

build-test.yml exposes a BeforeTestSteps parameter that runs after Prep Environment and before Run Tests — exactly the window required. All four cibuildwheel steps would move into that hook.

There is a latent bug in the way: live.tests.yml declares BeforeTestSteps as a parameter (line 20) but never forwards it to build-test.yml. Live-test callers that set it have their steps silently dropped today. jobs/ci.tests.yml forwards correctly. stages/archetype-sdk-tests.yml does pass the parameter into live.tests.yml, so this is reachable from real callers — stages/cosmos-sdk-client.yml is an existing consumer of BeforeTestSteps.

Wiring it through is ~3 lines and fixes that bug as a side effect. It should arguably be fixed regardless of what happens to this issue.

Orthogonal simplification, combinable with any of the above.

The CIBW_* environment variables are inert unless cibuildwheel actually runs. Setting CIBW_ARCHS, CIBW_SKIP, CIBW_TEST_SKIP, and CIBW_ENVIRONMENT_PASS_LINUX in a job that never invokes cibuildwheel has no effect whatsoever.

So the Configure cibuildwheel for dev requirement builds step could be made unconditional at zero risk. Only the two Windows NuGet steps genuinely need gating, because those cost wall-clock time and can fail.

That shrinks the surface the detection logic has to protect from four steps to two, and correspondingly reduces the cost of the detection being wrong: a false negative would then only mean a Windows job fetching CPython from a blocked endpoint (loud, obvious) rather than a silent misconfiguration.