Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

undefined behavior in gcd when parameters are minimum values of signed integers

Open
#33 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
45/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
cpp
Domain
backend

Research direction

Start in common_factor_rt.hpp at the gcd template around lines 444-451 and reproduce the call with both signed parameters set to std::numeric_limits::min(). Trace the recursive path and verify that the minimum-value case no longer reaches modulo by zero. Add or update regression coverage if the repository provides a relevant gcd test location.

Written by the indexing model from the issue text.

Description

This is being reported to me via a clang-analyzer check, and it refers to the gcd template function in common_factor_rt.hpp

Integer template parameter type is a signed integer, and both parameters ('a' and 'b') are std::numeric_limits<Integer>::min.

444 template <typename Integer>
445 inline BOOST_CXX14_CONSTEXPR Integer gcd(Integer const &a, Integer const &b) BOOST_GCD_NOEXCEPT(Integer)
446 {
447     if(a == (std::numeric_limits<Integer>::min)())
448        return a == static_cast<Integer>(0) ? gcd_detail::gcd_traits<Integer>::abs(b) : boost::integer::gcd(static_cast<Integer>(a % b), b);
449     else if (b == (std::numeric_limits<Integer>::min)())
450        return b == static_cast<Integer>(0) ? gcd_detail::gcd_traits<Integer>::abs(a) : boost::integer::gcd(a, static_cast<Integer>(b % a));
451     return gcd_detail::optimal_gcd_select(static_cast<Integer>(gcd_detail::gcd_traits<Integer>::abs(a)), static_cast<Integer>(gcd_detail::gcd_traits<Integer>::abs(b)));
452 }

In the initial invocation, the condition on 447 is true, and the condition on 448 is false. This leads to a recursive call to the function with parameters a=0 and b=std::numeric_limits<Integer>::min.

In the recursive call, the condition on 447 is false, the condition on 449 is true, and the condition on 450 is false. Another recursive call is on-deck with a=0 and b=b%a, But a is zero at this point. Anything mod 0 is undefined behavior.

My suggested fix is to check if a and b are the same value at the start of the function, and return a right away.

Dominant language
C++
Stars
19
Forks
58
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.