pyximport compilation failure tracebacks are excessively long.

Note: This is a very minor, quality-of-life annoyance.

I'm working on a relatively complex project that uses several cython modules, with additional pyxbld mediated c++ source files.

Right now, when I'm working on the C++ sources, cython compilation failures yield massively long tracebacks. The common failure output is 100+ lines.

This wouldn't be too much of a problem, if cython did not currently output the C++ compilation failure, and then the multiple python tracebacks it caused. As it is, basically all the information I need to actually fix the issue is inevitably scrolled off the screen.

Here is an example failure:

Error compiling Cython file:
------------------------------------------------------------
...

                # Queue structure:
                # [USB Thread] -> [Parse Thread] -> [Log Thread]

                self.usb_thread.insert_output_queue(self.parse_thread.get_queue_reference())
                self.parse_thread.insert_output_queue(self.log_thread.get_queue_reference())
                  ^
------------------------------------------------------------

Hardware/interface_wrapper.pyx:138:19: Object of type 'BatParserThread' has no attribute 'insert_output_queue'

Error compiling Cython file:
------------------------------------------------------------
...

                # Queue structure:
                # [USB Thread] -> [Parse Thread] -> [Log Thread]

                self.usb_thread.insert_output_queue(self.parse_thread.get_queue_reference())
                self.parse_thread.insert_output_queue(self.log_thread.get_queue_reference())
                                                                          ^
------------------------------------------------------------

Hardware/interface_wrapper.pyx:138:75: Cannot convert 'ReaderWriterQueue[UsbMessageRecord] &' to Python object
building 'Hardware.interface_wrapper' extension
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I. -I/usr/include/ -I/usr/lib/openmpi/include/ -I/usr/include/hdf5/serial -I/usr/include/python3.4m -c /home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.cpp -o /home/pi/.pyxbld/temp.linux-armv7l-3.4/home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.o -std=c++11 -O3 -Wall -Werror -lrt -lusb-1.0 -DLINUX
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
 #error Do not use this file, it is the result of a failed Cython compilation.
  ^
Traceback (most recent call last):
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 116, in _compile
    extra_postargs)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/usr/lib/python3.4/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/usr/lib/python3.4/distutils/spawn.py", line 162, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
    self.build_extension(ext)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "acquire_only.py", line 23, in <module>
    import Hardware.acq_hardware
  File "/home/pi/BatLogger/Interface/Hardware/acq_hardware.py", line 11, in <module>
    from . import interface_wrapper
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 445, in load_module
    language_level=self.language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 232, in load_module
    raise exc.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
    self.build_extension(ext)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile
    raise CompileError(msg)
ImportError: Building module Hardware.interface_wrapper failed: ["distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1\n"]

I don't see why the later tracebacks couldn't just re-raise the initial error, preventing the additional clutter. As it is, I tend to do my test-building python3 script_name.py 2>&1 | head -n 50, but shell hacks seem kind of crude.

Alternatively, why not pack the error messages from the compilation failure into the error? It already contains lines from further down the build process:

ImportError: Building module Hardware.interface_wrapper failed: ["distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1\n"]

It seems like if the g++ stderr was attached to the python error and percolated up through the handler chain, it could at least be output at the end of the traceback block, rather then before (for that matter, if you have a custom error with a overridden __repr__() and __str__(), does the interpreter use those when it's printed to the console)?