bazelbuild/bazel

Bazel uses exactly 25% more resources (as defined via resource_set) than given with --local_cpu_resources

Open

#17.230 aperta il 17 gen 2023

Vedi su GitHub
 (5 commenti) (0 reazioni) (0 assegnatari)Java (4465 fork)batch import
P3help wantedteam-Local-Exectype: bug

Metriche repository

Star
 (25.384 star)
Metriche merge PR
 (Merge medio 22g 20h) (77 PR mergiate in 30 g)

Descrizione

Description of the bug:

I have two actions that each declare CPU usage of 1000 by passing resource_set to actions.run_shell. bazel build --local_cpu_resources=1600 ... runs both actions in parallel. bazel build --local_cpu_resources=1599 ... runs them one after the other.

I would expect the cutoff to be between 1999 and 2000 instead.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

#!/bin/bash
set -euo pipefail


cat << 'EOF' > BUILD
load(":defs.bzl", "custom_bash_rule")

custom_bash_rule(
    name = "file1",
    command = "sleep 5; touch $OUTPUT",
    output = "file1",
)

custom_bash_rule(
    name = "file2",
    command = "sleep 5; touch $OUTPUT",
    output = "file2",
)
EOF


cat << 'EOF' > defs.bzl
# buildifier: disable=module-docstring
def _resource_set_callback(os_name, num_inputs):
    return {"cpu": 1000}

def _custom_bash_rule_impl(ctx):
    output_file = ctx.actions.declare_file(ctx.attr.output)

    ctx.actions.run_shell(
        command = "\n".join([
            "set -euo pipefail",
            'OUTPUT="' + output_file.path + '"',
            ctx.attr.command,
        ]),
        outputs = [output_file],
        resource_set = _resource_set_callback,
    )

    return [DefaultInfo(files = depset(direct = [output_file]))]

custom_bash_rule = rule(
    implementation = _custom_bash_rule_impl,
    attrs = {
        "command": attr.string(mandatory = True),
        "output": attr.string(mandatory = True),
    },
)
EOF


cat << 'EOF' > WORKSPACE
EOF


bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1600"
bazel build --local_cpu_resources=1600 ...

bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1599"
bazel build --local_cpu_resources=1599 ...

Output:

INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

Building with --local_cpu_resources=1600
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 7.840s, Critical Path: 5.07s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

Building with --local_cpu_resources=1599
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 12.873s, Critical Path: 10.08s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions

Which operating system are you running Bazel on?

Ubuntu 20.04

What is the output of bazel info release?

release 6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Guida contributor