How to set test timeout dynamically ?
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- cpp
- Domain
- testing-qa
Research direction
Start with the Boost.Test timeout documentation linked in the issue and compare the compile-time, runtime-variable, and command-line examples. Trace when BOOST_AUTO_TEST_CASE(timeout(get_timeout())) is evaluated versus when master_test_suite().argc and argv are initialized. Done means identifying a supported way to switch the timeout dynamically, or documenting the limitation and an appropriate alternative.
Written by the indexing model from the issue text.
Description
I am using Boost.Test version 1.87.0.
I am considering setting a uniform timeout for test cases. The purpose is to terminate a test case if it gets stuck due to some issue and proceed to the next test case.
However, I want to disable this timeout when debugging with a debugger such as gdb, as timeouts during debugging can interfere with the process.
According to the documentation:
https://www.boost.org/doc/libs/1_87_0/libs/test/doc/html/boost_test/testing_tools/timeout.html
it appears that timeouts can be specified using the described method.
First, I specified a timeout using a constant, following the example in the documentation.
- Passing the test timeout as a command-line argument
- Example using a compile-time constant
https://godbolt.org/z/9fneKoTvj
This resulted in the expected timeout.
However, to disable the timeout for debugging, recompilation is required.
Next, I attempted to specify a variable instead of a compile-time constant.
- Example using a runtime-determined variable
https://godbolt.org/z/GjzMrjPhG
This also resulted in the expected timeout.
To dynamically switch the timeout on or off and adjust its duration, I considered passing it as a command-line argument.
inline unsigned long get_timeout() {
std::cout << "In get_timeout(), argc:" << boost::unit_test::framework::master_test_suite().argc << std::endl;
if (boost::unit_test::framework::master_test_suite().argc > 1) {
return
std::stoul(
boost::unit_test::framework::master_test_suite().argv[1]
);
}
return 0;
}
- Example retrieving the value from a command-line argument
https://godbolt.org/z/6fTPjWbxj
However, in this case, the timeout did not take effect.
Upon analysis, it appears that:
BOOST_AUTO_TEST_CASE(t1, * boost::unit_test::timeout(get_timeout())) {
calls get_timeout(), but at this point,
boost::unit_test::framework::master_test_suite().argc and
boost::unit_test::framework::master_test_suite().argv
have not yet been initialized.
To confirm this, I added the following code inside the test case:
auto tout = get_timeout();
std::cout
<< "In test case argc: "
<< boost::unit_test::framework::master_test_suite().argc
<< " result of get_timeout(): "
<< tout
<< std::endl;
At this timing, argc and argv were correctly retrieved.
Is there a good way to dynamically switch the timeout?
- Dominant language
- C++
- Stars
- 213
- Forks
- 149
- Avg merge
- 18h 17m
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from boostorg/test
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 52/100
-
Conversion Warning with GCC15 when using operator<< within custom failure message and -WConversion Open
Difficulty 4/5 3-5 days Newbie friendliness 42/100
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·