`remoteActionBuildingSemaphore` slows down highly parallel remote builds

Description of the bug:

Highly parallel RBE builds are slower than they need to be because of remoteActionBuildingSemaphore.

Which category does this issue belong to?

Remote Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Have a build that can do 300+ actions at once, use --jobs=300, and a host machine with 24 or fewer threads. Build clean once with Bazel 7.0.0. Build clean once with remoteActionBuildingSemaphore usage commented out. Compare and see that the second build is faster.

Which operating system are you running Bazel on?

Ubuntu 22.04

What is the output of bazel info release?

development version

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

Built the release-7.0.0 branch and modified it per above.

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

https://github.com/bazelbuild/bazel/pull/17120 / https://github.com/bazelbuild/bazel/commit/4069a87611886532a19c7e558ab4c2f1e83f53f0 added the semaphore

Any other information, logs, or outputs that you want to share?

No response