undefined behavior in gcd when parameters are minimum values of signed integers
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 45/100
Research direction
Start in common_factor_rt.hpp at the gcd template around lines 444-451 and reproduce the call with both signed parameters set to std::numeric_limits::min(). Trace the recursive path and verify that the minimum-value case no longer reaches modulo by zero. Add or update regression coverage if the repository provides a relevant gcd test location.
Written by the indexing model from the issue text.
Description
This is being reported to me via a clang-analyzer check, and it refers to the gcd template function in common_factor_rt.hpp
Integer template parameter type is a signed integer, and both parameters ('a' and 'b') are std::numeric_limits<Integer>::min.
444 template <typename Integer>
445 inline BOOST_CXX14_CONSTEXPR Integer gcd(Integer const &a, Integer const &b) BOOST_GCD_NOEXCEPT(Integer)
446 {
447 if(a == (std::numeric_limits<Integer>::min)())
448 return a == static_cast<Integer>(0) ? gcd_detail::gcd_traits<Integer>::abs(b) : boost::integer::gcd(static_cast<Integer>(a % b), b);
449 else if (b == (std::numeric_limits<Integer>::min)())
450 return b == static_cast<Integer>(0) ? gcd_detail::gcd_traits<Integer>::abs(a) : boost::integer::gcd(a, static_cast<Integer>(b % a));
451 return gcd_detail::optimal_gcd_select(static_cast<Integer>(gcd_detail::gcd_traits<Integer>::abs(a)), static_cast<Integer>(gcd_detail::gcd_traits<Integer>::abs(b)));
452 }
In the initial invocation, the condition on 447 is true, and the condition on 448 is false. This leads to a recursive call to the function with parameters a=0 and b=std::numeric_limits<Integer>::min.
In the recursive call, the condition on 447 is false, the condition on 449 is true, and the condition on 450 is false. Another recursive call is on-deck with a=0 and b=b%a, But a is zero at this point. Anything mod 0 is undefined behavior.
My suggested fix is to check if a and b are the same value at the start of the function, and return a right away.
- Dominant language
- C++
- Stars
- 19
- Forks
- 58
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
AuTest Bug Tests
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/trafficserver#13714 ·
-
bug build
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
facebookincubator/velox#19143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
tenstorrent/tt-metal#57393 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
objectionary/eo-graphs#74 ·