bazelbuild/bazel

[7.6.1] `build --features=thin_lto` may fail with `error reading imports file .o.imports: Missing digest:`

Open

#26955 opened on Sep 11, 2025

View on GitHub
 (19 comments) (0 reactions) (0 assignees)Java (25,384 stars) (4,465 forks)batch import
P3help wantedpotential 7.x cherry-picksteam-Remote-Exectype: bug

Description

Description of the bug:

Hello recently I upgraded Bazel from 6.5.0 to 7.6.1 and ran into an error about remote cache, I tried --experimental_remote_cache_eviction_retries=5 but it doesn't help.

LTO Backend Compile bazel-out/arm-opt/bin/path/to/libbar.so.lto/bazel-out/arm-opt/bin/path/to/_objs/another/another.pic.o failed: \
error reading imports file /workdir/workspace/repo/.cache/execroot/my_workspace/bazel-out/arm-opt/bin/path/to/libbar.so.lto/bazel-out/arm-opt/bin/path/to/_objs/another/another.pic.o.imports: \
Missing digest: HASH/LENGTH for bazel-out/arm-opt/bin/path/to/libbar.so.lto/bazel-out/arm-opt/bin/path/to/_objs/another/another.pic.o

Details

  • --remote_download_outputs is the default value toplevel
  • and I also enabled --experimental_remote_cache_eviction_retries=5. So I once thought Bazel would auto retry for several times.
  • --remote_local_fallback=true
  • --incompatible_allow_tags_propagation is the default value true
cc_binary(
  name = "bar",
  linkshared = True,
  # I want to use "no-cache" to disable uploading libbar.so (which is too big: about 1.5GB),
  # however it seems `LTO indexing` inherits all the tags (though `LTO Backend Compile doesn't inherit them)
  tags = ["no-cache", "no-remote"],
  deps = [...],
)

And the command line was bazel build --remote_cache=http://my_cache //path/to:some_pkg

But the building log has nothing like ERROR: Build did NOT complete successfully\nFound transient remote cache error, retrying the build...

After reading source code of version 7.6.1, I think it's a bug of Bazel:

  • experimental_remote_cache_eviction_retries is only checked by
    • RemoteSpawnRunner::exec -> execLocallyAndUploadOrFail -> handleError
    • LocalSpawnRunner::exec -> prefetchInputsAndWait -> prefetchInputs
    • BlazeCommandDispatcher::exec -> execExclusively
  • but, error reading imports file is in LtoBackendAction::discoverInputs,
    • called by ActionExecutionFunction::checkCacheAndExecuteIfNeeded
    • called by ActionExecutionFunction::compute
    • called by AbstractParallelEvaluator::run
    • (there're too many .run(), not surce where to call it)
  • Then the LtoBackendAction::discoverInputs is not protected by remoteRetryOnTransientCacheError

Which category does this issue belong to?

Remote Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

Ubuntu 20.04, x86_64

What is the output of bazel info release?

release 7.6.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

(A private repo)

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

Not found any LTO+remote-cache issue.

Any other information, logs, or outputs that you want to share?

No response

Contributor guide