bazel 5.x.x crashes if uncompressed CAS is uploaded to remote cache

Description of the problem / feature request:

When experimenting with compression (https://github.com/bazelbuild/bazel/pull/14041), I ran into crashes if the server isn't on the same page as the client. I think these are edge cases as in this case, remote cache server hasn't fully implemented compression support, but it's possible bazel may want to guard against these and not crash fatally since we are just using remote cache.

Feature requests: what underlying problem are you trying to solve with this feature?

Running with compression flag: bazel build //... --experimental_remote_cache_compression --remote_upload_local_results=true

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

If server claims to support zstd but doesn't, client (bazel) works fine. However, issue can be reproduced if we try and upload something that is compressed.

What operating system are you running Bazel on?

Mac OS.

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

origin/release-5.0.0rc3, 5.1.0

Any other information, logs, or outputs that you want to share?

This stack is if server throws an error (expected), which I think should be warning and let bazel build locally instead of crashing.

...	
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: compressed-blobs/zstd/f7675cc879aba2e38d0d059ad60e3ed19d3d062c3fdb9b2fd37a2a8c8ed848ee/202: Url path not recognized
		at io.grpc.Status.asRuntimeException(Status.java:535)
		... 14 more
[96 / 99] GoCompilePkg src/code.uber.internal/base/io/fileutil/go_default_test_test.external.a; 0s remote-cache
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//src/code.uber.internal/base/io/fileutil:go_default_test, config=BuildConfigurationValue.Key[1bb1b4d6ce810b0badbe269c1bb76d3539cd5517823c629b93b9bf6dc6f01409]}, actionIndex=1}' (requested by nodes 'ArtifactNestedSetKey{rawChildren=[File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_davecgh_go_spew/spew/spew.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_pmezard_go_difflib/difflib/difflib.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/in_gopkg_yaml_v3/yaml_v3.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_stretchr_testify/assert/assert.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_stretchr_testify/require/require.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/org_uber_go_atomic/atomic.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/org_uber_go_multierr/multierr.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]src/code.uber.internal/base/io/fileutil/go_default_test.internal.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]src/code.uber.internal/base/io/fileutil/go_default_test_test.external.a]}', ...)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:674)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	

and here is if server incorrectly passes back uncompressed CAS

	Caused by: java.io.IOException: Decompression error: Unknown frame descriptor
		at com.github.luben.zstd.ZstdInputStreamNoFinalizer.readInternal(ZstdInputStreamNoFinalizer.java:171)
		at com.github.luben.zstd.ZstdInputStreamNoFinalizer.read(ZstdInputStreamNoFinalizer.java:123)
		at com.github.luben.zstd.ZstdInputStream.read(ZstdInputStream.java:87)
		at com.google.protobuf.ByteString.readChunk(ByteString.java:540)
		at com.google.protobuf.ByteString.readFrom(ByteString.java:517)
		at com.google.protobuf.ByteString.readFrom(ByteString.java:485)
		at com.google.devtools.build.lib.remote.zstd.ZstdDecompressingOutputStream.write(ZstdDecompressingOutputStream.java:60)
		at com.google.devtools.build.lib.remote.zstd.ZstdDecompressingOutputStream.write(ZstdDecompressingOutputStream.java:54)
		at com.google.protobuf.ByteString$LiteralByteString.writeTo(ByteString.java:1381)
		at com.google.devtools.build.lib.remote.GrpcCacheClient$1.onNext(GrpcCacheClient.java:389)
		... 12 more
[96 / 99] GoCompilePkg src/code.uber.internal/base/io/fileutil/go_default_test_test.external.a; 0s remote-cache
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//src/code.uber.internal/base/io/fileutil:go_default_test, config=BuildConfigurationValue.Key[1bb1b4d6ce810b0badbe269c1bb76d3539cd5517823c629b93b9bf6dc6f01409]}, actionIndex=1}' (requested by nodes 'ArtifactNestedSetKey{rawChildren=[File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_davecgh_go_spew/spew/spew.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_pmezard_go_difflib/difflib/difflib.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/in_gopkg_yaml_v3/yaml_v3.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_stretchr_testify/assert/assert.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/com_github_stretchr_testify/require/require.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/org_uber_go_atomic/atomic.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]external/org_uber_go_multierr/multierr.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]src/code.uber.internal/base/io/fileutil/go_default_test.internal.a, File:[[<execution_root>]bazel-out/darwin-fastbuild/bin]src/code.uber.internal/base/io/fileutil/go_default_test_test.external.a]}', ...)