Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

paramFile.c byte-swaps through unions (strict-aliasing UB): arParamLoad() returns garbage distortion with GCC -O3

Open
  • Is something wrong?
#101 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
57/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
c, cpp
Domain
backend, performance

Research direction

Start with lib/SRC/AR/paramFile.c, read byteSwapDouble(), byteSwapInt(), and byteswap(), then reproduce the reported arParamLoad() result with GCC at -O3. Check that the byte-swapped dist_factor values match the camera file; the report also identifies an MSVC numeric_limits<int>::max() issue in ARToolKitNFTCore::setup(), so check that expression and its Windows build behavior too.

Written by the indexing model from the issue text.

Description

What happens

lib/SRC/AR/paramFile.c byte-swaps the camera parameters through pointers to unions:

static void byteSwapDouble( const double *from, double *to )
{
    SwapDoubleT   *w1, *w2;
    ...
    w1 = (SwapDoubleT *)from;
    w2 = (SwapDoubleT *)to;
    for( i = 0; i < 8; i++ ) {
        w2->y[i] = w1->y[7-i];
    }
}

byteSwapInt() does the same with SwapIntT. Accessing a double or an int object through a pointer to a union type breaks C's strict-aliasing rule.

GCC 13 at -O3 vectorises the dist_factor loop of byteswap(). arParamLoad() then returns garbage distortion factors (around 1e-310) instead of the file's values, while mat is right. AR2 projects the features with that distortion, and the NFT tracking loses every marker right after KPM finds it: ar2TrackingMod returns -3.

Evidence

From webarkit/jsartoolkitNFT#693, Ubuntu 24.04, gcc 13.3:

  • a small program that calls arParamLoad("camera_para.dat") and prints dist_factor gives the right values at -O2 and garbage at -O3;
  • at -O3, adding -fno-strict-aliasing, or disabling vectorisation (-fno-tree-loop-vectorize -fno-tree-slp-vectorize), gives the right values again;
  • the jsartoolkitNFT Python bindings were built at -O3 on Linux, which is the level of the Pythons from actions/setup-python, and lost every marker. Bisecting with per-file -O2 pointed to paramFile.c alone.

Emscripten (clang) and MSVC builds are not affected.

Suggested fix

Swap through unsigned char pointers, which may alias any object:

static void byteSwapDouble( const double *from, double *to )
{
    const unsigned char *f = (const unsigned char *)from;
    unsigned char       *t = (unsigned char *)to;
    int i;
    for( i = 0; i < 8; i++ ) t[i] = f[7-i];
}

Apply the same change to byteSwapInt(), and to the commented-out byteSwapFloat(). Then drop the three Swap*T unions. A native test that loads camera_para.dat and checks a known dist_factor value, built at -O3, would catch a regression.

A second, smaller item: MSVC and std::numeric_limits<int>::max()

ARToolKitNFTCore::setup() uses std::numeric_limits<int>::max(). On Windows, <AR/config.h> includes <windows.h>, whose max macro breaks that expression (error C2589) unless NOMINMAX is defined. Writing it as (std::numeric_limits<int>::max)() makes it compile either way.

jsartoolkitNFT's Python build currently works around both problems: -fno-strict-aliasing for paramFile.c, and NOMINMAX on Windows.

Dominant language
C++
Stars
21
Forks
8
Avg merge
7h 30m
Merged PRs (30d)
25

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from webarkit/WebARKitLib

All issues in webarkit/WebARKitLib

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.