cstdlib shipped with gcc 6, 7, and 8 fail to find stdlib.h in standalone spawn strategy which causes build failures

ATTENTION! Please read and follow:

• if this is a question about how to build / test / query / deploy using Bazel, ask it on StackOverflow instead: https://stackoverflow.com/questions/tagged/bazel
• if this is a discussion starter, send it to bazel-discuss@googlegroups.com
• if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

It seems that due to a change made in cstdlib starting with gcc 6, the use of -isystem command line options causes the system include order to change and thus c++ code which uses cstdlib will fail to build if the spawn strategy does not use the sandbox. See the bottom of the report for the full description of what is going on. The issue in question is documented here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70936

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Setup a simple C++ project with a single file and have cstdlib included either directly or indirectly. Perform a bazel build with --spawn_strategy=standalone and the error should appear.

What operating system are you running Bazel on?

Observed with gcc 6, 7, and 8 on ubuntu 16.04, ubuntu 19.04, Linux Mint 19.1, and Gentoo Linux.

What's the output of bazel info release?

Linux Mint 19.1: release 0.24.1 Gentoo Linux: release 0.24.1- (@non-git) [Built via emerge] Ubuntu 16.04: release 0.25.2, release 0.22.0, and release 0.21.0 Ubuntu 19.04: release 0.25.2

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

Unsure how to answer this, which repo? Some of it is proprietary

Have you found anything relevant by searching the web?

The only thing relevant is the bugzilla link (which was found by a coworker) I pasted above, here it is again for completeness: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70936

Any other information, logs, or outputs that you want to share?

So for brevity, I will do a brain dump from this point forward. This is a copy (cleaned up to strip anything proprietary) from our internal JIRA tracker. We have encountered this issue with a customer. Before I describe the issue and background, please don't hesitate to ask questions or request clarification.

So to start with, we encountered the error originally trying to manually run a compile command line generated by bazel. Here is an example of the error we saw:

$ /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' -fPIC -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/system_project -iquote bazel-out/k8-fastbuild/bin/external/system_project -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools -isystem external/system_project/include -isystem bazel-out/k8-fastbuild/bin/external/system_project/include -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c main/hello-world.cc
In file included from /usr/include/c++/8/ext/string_conversions.h:41,
                 from /usr/include/c++/8/bits/basic_string.h:6400,
                 from /usr/include/c++/8/string:52,
                 from main/hello-world.cc:2:
/usr/include/c++/8/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
 #include_next <stdlib.h>
               ^~~~~~~~~~
compilation terminated.

This error stems from the above bugzilla link where GCC replaced #include <stdlib.h> with #include_next <stdlib.h> in GCC 6 and above in the file cstdlib. This error baffled us for a while because it does not manifest when building with bazel itself. At first we thought there may be a bazel specific hack taking place but as you'll see the actual issue is far simpler.

When I run our example project through bazel on my Gentoo Linux machine with sandbox debugging I see the following information:

jscoggins@cobra ~/tmp/CPP-43460 
$ bazel build //... -s --sandbox_debug --copt=-v
<Information cut here>
INFO: From Compiling main/hello-world.cc:
src/main/tools/linux-sandbox.cc:154: linux-sandbox-pid1 has PID 7801
src/main/tools/linux-sandbox-pid1.cc:175: working dir: /home/jscoggins/.cache/bazel/_bazel_jscoggins/43973faf65a1791ca71ea4969beeaceb/sandbox/linux-sandbox/1/execroot/__main__
src/main/tools/linux-sandbox-pid1.cc:194: writable: /home/jscoggins/.cache/bazel/_bazel_jscoggins/43973faf65a1791ca71ea4969beeaceb/sandbox/linux-sandbox/1/execroot/__main__
src/main/tools/linux-sandbox-pid1.cc:194: writable: /tmp
src/main/tools/linux-sandbox-pid1.cc:194: writable: /dev/shm
<More diagnostic information and rest of build>

In this case we are running with the sandbox enabled and the working directory points to sandbox directory as it should. This is correct, everything looks fine, and we are successful in building. Here is where the issue actual arises, I have added -v to the command lines so I can see what gcc is doing. Here is the snipped log for that:

jscoggins@cobra ~/tmp/CPP-43460 
$ bazel build //... -s --sandbox_debug --copt=-v
<SNIPPED FOR BREVITY>
src/main/tools/linux-sandbox.cc:154: linux-sandbox-pid1 has PID 7801
<SNIPPED FOR BREVITY>
<START GCC VERBOSE INFORMATION>
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-7.3.0-r3/work/gcc-7.3.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/7.3.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 7.3.0-r3 p1.4' --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libmudflap --disable-libssp --disable-libcilkrts --disable-libmpx --enable-vtable-verify --enable-libvtv --enable-lto --without-isl --enable-libsanitizer --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (Gentoo 7.3.0-r3 p1.4) 
COLLECT_GCC_OPTIONS='-U' '_FORTIFY_SOURCE' '-fstack-protector' '-Wall' '-Wunused-but-set-parameter' '-Wno-free-nonheap-object' '-fno-omit-frame-pointer' '-std=c++11' '-MD' '-MF' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d' '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-fPIC' '-iquote' '.' '-iquote' 'bazel-out/k8-fastbuild/genfiles' '-iquote' 'bazel-out/k8-fastbuild/bin' '-iquote' 'external/system_project' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/system_project' '-iquote' 'bazel-out/k8-fastbuild/bin/external/system_project' '-iquote' 'external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/bin/external/bazel_tools' '-isystem' 'external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/genfiles/external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/bin/external/system_project/include' '-v' '-fno-canonical-system-headers' '-Wno-builtin-macro-redefined' '-D' '__DATE__="redacted"' '-D' '__TIMESTAMP__="redacted"' '-D' '__TIME__="redacted"' '-c' '-o' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-pc-linux-gnu/7.3.0/cc1plus -quiet -v -MD bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d -MF bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d -MQ bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -D_GNU_SOURCE -U _FORTIFY_SOURCE -D __DATE__="redacted" -D __TIMESTAMP__="redacted" -D __TIME__="redacted" -iquote . -iquote bazel-out/k8-fastbuild/genfiles -iquote bazel-out/k8-fastbuild/bin -iquote external/system_project -iquote bazel-out/k8-fastbuild/genfiles/external/system_project -iquote bazel-out/k8-fastbuild/bin/external/system_project -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/genfiles/external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools -isystem external/system_project/include -isystem bazel-out/k8-fastbuild/genfiles/external/system_project/include -isystem bazel-out/k8-fastbuild/bin/external/system_project/include main/hello-world.cc -quiet -dumpbase hello-world.cc -mtune=generic -march=x86-64 -auxbase-strip bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -Wno-builtin-macro-redefined -std=c++11 -version -fstack-protector -fno-omit-frame-pointer -frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -fPIC -fno-canonical-system-headers -o /tmp/ccWIeYo2.s
GNU C++11 (Gentoo 7.3.0-r3 p1.4) version 7.3.0 (x86_64-pc-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.6, MPC version 1.0.3, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles"
ignoring nonexistent directory "external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/bazel_tools"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/bazel_tools"
#include "..." search starts here:
 .
 bazel-out/k8-fastbuild/bin
 external/bazel_tools
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
 /usr/include
End of search list.
GNU C++11 (Gentoo 7.3.0-r3 p1.4) version 7.3.0 (x86_64-pc-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.6, MPC version 1.0.3, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 3a6a2fe9d2bc99e515e9d68cc7f74eee
COLLECT_GCC_OPTIONS='-U' '_FORTIFY_SOURCE' '-fstack-protector' '-Wall' '-Wunused-but-set-parameter' '-Wno-free-nonheap-object' '-fno-omit-frame-pointer' '-std=c++11' '-MD' '-MF' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d' '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-fPIC' '-iquote' '.' '-iquote' 'bazel-out/k8-fastbuild/genfiles' '-iquote' 'bazel-out/k8-fastbuild/bin' '-iquote' 'external/system_project' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/system_project' '-iquote' 'bazel-out/k8-fastbuild/bin/external/system_project' '-iquote' 'external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/bin/external/bazel_tools' '-isystem' 'external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/genfiles/external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/bin/external/system_project/include' '-v' '-fno-canonical-system-headers' '-Wno-builtin-macro-redefined' '-D' '__DATE__="redacted"' '-D' '__TIMESTAMP__="redacted"' '-D' '__TIME__="redacted"' '-c' '-o' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/bin/as -v --64 -o bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o /tmp/ccWIeYo2.s
GNU assembler version 2.30.0 (x86_64-pc-linux-gnu) using BFD version (Gentoo 2.30 p5) 2.30.0
COMPILER_PATH=/usr/libexec/gcc/x86_64-pc-linux-gnu/7.3.0/:/usr/libexec/gcc/x86_64-pc-linux-gnu/7.3.0/:/usr/libexec/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/:/usr/lib/gcc/x86_64-pc-linux-gnu/:/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/:/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/lib/:/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-U' '_FORTIFY_SOURCE' '-fstack-protector' '-Wall' '-Wunused-but-set-parameter' '-Wno-free-nonheap-object' '-fno-omit-frame-pointer' '-std=c++11' '-MD' '-MF' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d' '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-fPIC' '-iquote' '.' '-iquote' 'bazel-out/k8-fastbuild/genfiles' '-iquote' 'bazel-out/k8-fastbuild/bin' '-iquote' 'external/system_project' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/system_project' '-iquote' 'bazel-out/k8-fastbuild/bin/external/system_project' '-iquote' 'external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/bin/external/bazel_tools' '-isystem' 'external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/genfiles/external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/bin/external/system_project/include' '-v' '-fno-canonical-system-headers' '-Wno-builtin-macro-redefined' '-D' '__DATE__="redacted"' '-D' '__TIMESTAMP__="redacted"' '-D' '__TIME__="redacted"' '-c' '-o' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-mtune=generic' '-march=x86-64'
<END GCC VERBOSE INFORMATION>
<SNIPPED FOR BREVITY>
src/main/tools/linux-sandbox.cc:204: child exited normally with exitcode 0
<SNIPPED FOR BREIVTY>
INFO: Elapsed time: 0.419s, Critical Path: 0.21s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed successfully, 6 total actions

The -isystem options added by bazel are not used during the build and gcc is free to put /usr/include at the end of the list. Thus the #include_next <stdlib.h> directive in cstdlib works because /usr/include comes last in the search order. Here is that portion of the above log in detail:

ignoring nonexistent directory "external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles"
ignoring nonexistent directory "external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/bazel_tools"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/bazel_tools"
#include "..." search starts here:
 .
 bazel-out/k8-fastbuild/bin
 external/bazel_tools
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
 /usr/include
End of search list.

When I disable the sandbox I am able to trigger the compilation failure shown previously:

jscoggins@cobra ~/tmp/CPP-43460 
$ bazel build //... -s --sandbox_debug --copt=-v --spawn_strategy=standalone
INFO: Analysed target //main:hello-world (10 packages loaded, 65 targets configured).
INFO: Found 1 target...
SUBCOMMAND: # //main:hello-world [action 'Compiling main/hello-world.cc']
(cd /home/jscoggins/.cache/bazel/_bazel_jscoggins/43973faf65a1791ca71ea4969beeaceb/execroot/__main__ && \
  exec env - \
    LD_LIBRARY_PATH=:/opt/emul-linux-x86-java-1.7.0.55/lib/i386:/opt/emul-linux-x86-java-1.7.0.55/lib/i386/server:/opt/emul-linux-x86-java-1.7.0.55/lib/i386:/opt/emul-linux-x86-java-1.7.0.55/lib/i386/server \
    PATH=/usr/lib/llvm/8/bin:/usr/lib/llvm/7/bin:/usr/lib/llvm/6/bin:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/games/bin:/home/jscoggins/bin:/home/jscoggins/sys/inferno-os/Linux/386/bin:/home/jscoggins/bin:/home/jscoggins/sys/maven/3.2.3/bin:/home/jscoggins/sys/go/bin:/home/jscoggins/dev/go/bin:/home/jscoggins/sys/parasoft//insure/Linux/amd64/7.4.5/bin:/home/jscoggins/sys/inferno-os//Linux/386/bin:/home/jscoggins/sys/arcanist/bin:/home/jscoggins/sys/toolchains/i960-elf-gcc-3.4.6/bin:/home/jscoggins/sys/toolchains/i960-coff-gcc-3.4.6/bin \
    PWD=/proc/self/cwd \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' -fPIC -iquote . -iquote bazel-out/k8-fastbuild/genfiles -iquote bazel-out/k8-fastbuild/bin -iquote external/system_project -iquote bazel-out/k8-fastbuild/genfiles/external/system_project -iquote bazel-out/k8-fastbuild/bin/external/system_project -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/genfiles/external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools -isystem external/system_project/include -isystem bazel-out/k8-fastbuild/genfiles/external/system_project/include -isystem bazel-out/k8-fastbuild/bin/external/system_project/include -v -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c main/hello-world.cc -o bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o)
ERROR: /home/jscoggins/tmp/CPP-43460/main/BUILD:1:1: C++ compilation of rule '//main:hello-world' failed (Exit 1)
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-7.3.0-r3/work/gcc-7.3.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/7.3.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/7.3.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 7.3.0-r3 p1.4' --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libmudflap --disable-libssp --disable-libcilkrts --disable-libmpx --enable-vtable-verify --enable-libvtv --enable-lto --without-isl --enable-libsanitizer --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (Gentoo 7.3.0-r3 p1.4) 
COLLECT_GCC_OPTIONS='-U' '_FORTIFY_SOURCE' '-fstack-protector' '-Wall' '-Wunused-but-set-parameter' '-Wno-free-nonheap-object' '-fno-omit-frame-pointer' '-std=c++11' '-MD' '-MF' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d' '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-fPIC' '-iquote' '.' '-iquote' 'bazel-out/k8-fastbuild/genfiles' '-iquote' 'bazel-out/k8-fastbuild/bin' '-iquote' 'external/system_project' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/system_project' '-iquote' 'bazel-out/k8-fastbuild/bin/external/system_project' '-iquote' 'external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/genfiles/external/bazel_tools' '-iquote' 'bazel-out/k8-fastbuild/bin/external/bazel_tools' '-isystem' 'external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/genfiles/external/system_project/include' '-isystem' 'bazel-out/k8-fastbuild/bin/external/system_project/include' '-v' '-fno-canonical-system-headers' '-Wno-builtin-macro-redefined' '-D' '__DATE__="redacted"' '-D' '__TIMESTAMP__="redacted"' '-D' '__TIME__="redacted"' '-c' '-o' 'bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-pc-linux-gnu/7.3.0/cc1plus -quiet -v -MD bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d -MF bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d -MQ bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -D_GNU_SOURCE -U _FORTIFY_SOURCE -D __DATE__="redacted" -D __TIMESTAMP__="redacted" -D __TIME__="redacted" -iquote . -iquote bazel-out/k8-fastbuild/genfiles -iquote bazel-out/k8-fastbuild/bin -iquote external/system_project -iquote bazel-out/k8-fastbuild/genfiles/external/system_project -iquote bazel-out/k8-fastbuild/bin/external/system_project -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/genfiles/external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools -isystem external/system_project/include -isystem bazel-out/k8-fastbuild/genfiles/external/system_project/include -isystem bazel-out/k8-fastbuild/bin/external/system_project/include main/hello-world.cc -quiet -dumpbase hello-world.cc -mtune=generic -march=x86-64 -auxbase-strip bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -Wno-builtin-macro-redefined -std=c++11 -version -fstack-protector -fno-omit-frame-pointer -frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o -fPIC -fno-canonical-system-headers -o /tmp/ccHqu4Yr.s
GNU C++11 (Gentoo 7.3.0-r3 p1.4) version 7.3.0 (x86_64-pc-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.6, MPC version 1.0.3, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring duplicate directory "/usr/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/bazel_tools"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/bazel_tools"
#include "..." search starts here:
 .
 bazel-out/k8-fastbuild/genfiles
 bazel-out/k8-fastbuild/bin
 external/system_project
 external/bazel_tools
#include <...> search starts here:
 external/system_project/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
End of search list.
GNU C++11 (Gentoo 7.3.0-r3 p1.4) version 7.3.0 (x86_64-pc-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.6, MPC version 1.0.3, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 3a6a2fe9d2bc99e515e9d68cc7f74eee
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/ext/string_conversions.h:41:0,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/bits/basic_string.h:6349,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/string:52,
                 from main/hello-world.cc:2:
/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
 #include_next <stdlib.h>
               ^~~~~~~~~~
compilation terminated.
Target //main:hello-world failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.197s, Critical Path: 0.03s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

You'll notice that the search list has changed because the working directory has changed to /home/jscoggins/.cache/bazel/_bazel_jscoggins/43973faf65a1791ca71ea4969beeaceb/execroot/__main__ from the sandbox directory. The -isystem command line options are now used and change the search order when dealing with includes. Here is that portion of the above log:

 ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring duplicate directory "/usr/include"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/system_project"
ignoring nonexistent directory "bazel-out/k8-fastbuild/genfiles/external/bazel_tools"
ignoring nonexistent directory "bazel-out/k8-fastbuild/bin/external/bazel_tools"
#include "..." search starts here:
 .
 bazel-out/k8-fastbuild/genfiles
 bazel-out/k8-fastbuild/bin
 external/system_project
 external/bazel_tools
#include <...> search starts here:
 external/system_project/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
End of search list.

Since external/system_project/include refers to /usr/include gcc determines that /usr/include is a duplicate and does not add it to the end of the list. When cstdlib's #include_next <stdlib.h> directive is evaluated, the #include_next causes external/system_project/include to be ignored in the lookup list. This is what causes the lookup failure to occur and thus the compilation error as well. This locks bazel out of doing standalone builds with gcc 6 and up when compiling C++ code which need cstdlib.

To solve this problem we are eliminating the option -isystem external/system_project/include manually before performing analysis with our external analysis tools.