Test structure is stressful in terms of compiler memory consumption

Open
#321 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
48/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Stale
Tech stack
cpp
Domain
testing-qa

Research direction

Start with test/merge_separate.cpp at lines 21-47, where the long list of static_asserts and instantiations is shown. Review related tests for the same structure, then use MSVC with /d1reportMemorySummary to compare compiler peak working set size. Done means the affected test structure is refactored into separate functions and still passes.

Written by the indexing model from the issue text.

Description

Hi Boost.Parser devs! I'm the primary maintainer of MSVC's STL, and our compiler team has encountered issues with recent MSVC STL changes (specifically our implementation of C++23 reference_meows_from_temporary in tuple's constructors) causing Boost.Parser tests to have excessive compiler memory consumption, reaching out-of-memory for our accursed 32-bit x86-hosted compiler. (We regularly test open-source projects, especially Boost, with development builds of the compiler and libraries to find and fix regressions before they can ship and affect you.)

This is not a bug in your tests so you can close this issue if you like. However, I wanted to at least mention that the specific structure of the tests is leading to this excessive compiler memory consumption and a simple refactoring would make them less likely to OOM, and (as a benefit to you) probably a bit faster to compile.

The issue is when main() or any function has a long list of static_asserts and instantiations like this:

https://github.com/boostorg/parser/blob/293f5e1e3f491f9219e69269f464da6f2ab9ff94/test/merge_separate.cpp#L21-L47

Even though this code is in separate scopes, they're still within the same function, so the compiler doesn't reuse memory effectively. If these were in separate functions, then I believe the compiler memory consumption would be limited. (I don't have an exhaustive list of tests affected, this is just the one in the error message I saw.)

If you're interested in doing this, the undocumented but very useful compiler option /d1reportMemorySummary will report memory consumption. Here's an example showing how <print> is a chonker compared to <cstdio>, "Peak working set size" is the value of interest:

C:\Temp>type meow.cpp
#ifdef USE_CXX23
#include <print>
#else
#include <cstdio>
#endif

int main() {
#ifdef USE_CXX23
    std::println("Hello C++23 world!");
#else
    std::puts("Hello C++98 world!");
#endif
}

C:\Temp>cl /EHsc /nologo /W4 /MTd /Od /std:c++14 /d1reportMemorySummary meow.cpp && meow
meow.cpp
Persistent heap 1 size: 5424544
Persistent heap 2 size: 1255624
Peak working set size: 20422656
Hello C++98 world!

C:\Temp>cl /EHsc /nologo /W4 /MTd /Od /std:c++latest /DUSE_CXX23 /d1reportMemorySummary meow.cpp && meow
meow.cpp
Persistent heap 1 size: 84938280
Persistent heap 2 size: 4237816
Peak working set size: 113741824
Hello C++23 world!
Dominant language
C++
Stars
182
Forks
28
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from boostorg/parser

All issues in boostorg/parser

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.