emitRestoreFlagsFromStackSlot restores a corrupted RFLAGS image (x86-64)

Start with dyninstAPI/src/emit-x86.C and trace EmitterAMD64::emitRestoreFlagsFromStackSlot back to the saved-flags value produced by dyninstAPI/src/registerSpace/registerSpace.C::saveVolatileRegisters. Check the ifMCOp path in dyninstAPI/src/ASTs/operatorAST.C and the emitCSload path for scas/cmps; compare how flags are saved and restored, then run the supplied x86-64 reproducer. Done means the conditional case no longer crashes and counts only executed accesses, while the plain control still runs.

Summary

On x86-64, inserting a BPatch_ifMachineConditionExpr at a conditional
memory access (e.g. a cmov that reads memory) corrupts the process's flags
and typically crashes the target (SIGBUS or SIGTRAP), or silently runs the
snippet on the wrong iterations. The same corruption path is reachable from the
scas/cmps memory-access emulation (emitCSload with IA32_ESCAS/IA32_ECMPS).

Root cause

To run its body only when the instrumented conditional access actually
executed, ifMachineCondition restores the original flags mid-trampoline via
EmitterAMD64::emitRestoreFlagsFromStackSlot (dyninstAPI/src/emit-x86.C),
which emits:

push (<flags-slot>)     ; push the saved-flags stack slot
popfq                   ; RFLAGS <- that slot
jCC  ...                ; branch on the "restored" condition

But that slot was written by registerSpace::saveVolatileRegisters
(dyninstAPI/src/registerSpace/registerSpace.C) as a lahf (→%ah) + seto %al
value pushed as %rax — not an RFLAGS bit-image. (EmitterAMD64::emitPushFlags,
the only real pushfq emitter, is dead code on x86 — it has no callers — so
nothing ever deposits a valid image.) popfq therefore loads a scrambled value;
decoding the lahf/seto %rax layout back through popfq gives:

RFLAGS bit	loaded from	should be
CF (0)	original OF	CF
PF/AF/ZF/SF	0	their real values
TF (8)	original CF	0 → spurious single-step (SIGTRAP)
DF (10)	original PF	preserved DF
OF (11)	0	OF
AC (18) region	garbage	0 → alignment-check (SIGBUS) on the next misaligned access

So the following jCC branches on garbage, and stray control/system flags get
set. Observed as a SIGBUS crash below; on other inputs it manifests as a
spurious SIGTRAP (TF) or as the body running on the wrong iterations.

The save/restore representations simply disagree: the producer writes a
lahf/seto %rax value, the consumer reads it back with popfq as if it were
a full flags image.

Reproducer

Verified on x86-64 (gcc 15, binary rewriter). A cmovg that reads memory is the
conditional access; instrumenting it with ifMachineCondition crashes, while the
identical body without the wrapper (the control) runs cleanly — isolating the
crash to emitRestoreFlagsFromStackSlot.

mutatee.c

#include <stdio.h>

volatile long mem = 0x42;
long access_count = 0;            /* incremented by instrumentation */

__attribute__((noinline)) long cond_access(long x) {
    long r = -1;
    __asm__ volatile (
        "test %1, %1\n\t"         /* flags from x; test clears CF */
        "cmovg %2, %0\n\t"        /* r = (x > 0) ? mem : r  -- conditional read */
        : "+r"(r) : "r"(x), "m"(mem) : "cc");
    return r;
}

int main(void) {
    long expected = 0;
    for (long i = -5; i <= 5; i++) {          /* condition true for i = 1..5 */
        if (cond_access(i) == mem) expected++;
    }
    printf("expected conditional-access count = %ld\n", expected);   /* 5 */
    printf("instrumented access_count        = %ld\n", access_count);
    return 0;
}

mutator.cpp (binary rewriter; default inserts ifMachineCondition, arg
plain inserts the bare body as a control)

#include "BPatch.h"
#include "BPatch_binaryEdit.h"
#include "BPatch_image.h"
#include "BPatch_function.h"
#include "BPatch_point.h"
#include "BPatch_snippet.h"
#include "BPatch_memoryAccess_NP.h"
#include <cstdio>
#include <vector>
#include <set>
#include <string>
using namespace Dyninst;

int main(int argc, char** argv) {
    if (argc < 3) { fprintf(stderr, "usage: %s <in> <out> [plain]\n", argv[0]); return 2; }
    BPatch bpatch;
    BPatch_binaryEdit* be = bpatch.openBinary(argv[1], false);
    if (!be) { fprintf(stderr, "openBinary failed\n"); return 1; }
    BPatch_image* img = be->getImage();

    std::vector<BPatch_function*> fns;
    img->findFunction("cond_access", fns);
    if (fns.empty()) { fprintf(stderr, "cond_access not found\n"); return 1; }

    BPatch_variableExpr* cnt = img->findVariable("access_count");
    if (!cnt) { fprintf(stderr, "access_count not found\n"); return 1; }

    std::set<BPatch_opCode> ops; ops.insert(BPatch_opLoad); ops.insert(BPatch_opStore);
    std::vector<BPatch_point*>* pts = fns[0]->findPoint(ops);
    if (!pts || pts->empty()) { fprintf(stderr, "no memory-access points\n"); return 1; }

    // body:  access_count = access_count + 1
    BPatch_arithExpr incr(BPatch_assign, *cnt,
                          BPatch_arithExpr(BPatch_plus, *cnt, BPatch_constExpr(1)));

    // default "cond" wraps the body in ifMachineCondition (triggers the bug);
    // "plain" inserts the bare body (no flags restore) as a control.
    bool cond = !(argc >= 4 && std::string(argv[3]) == "plain");
    BPatch_ifMachineConditionExpr condSnip(incr);
    BPatch_snippet& snip = cond ? static_cast<BPatch_snippet&>(condSnip)
                                : static_cast<BPatch_snippet&>(incr);

    // Instrument only genuinely-conditional accesses (the cmovg).
    int n = 0;
    for (auto p : *pts) {
        const BPatch_memoryAccess* ma = p->getMemoryAccess();
        if (!ma || ma->conditionCode_NP() < 0) continue;
        if (be->insertSnippet(snip, *p)) n++;
    }
    fprintf(stderr, "inserted %s snippet at %d conditional point(s)\n",
            cond ? "ifMachineCondition" : "PLAIN(control)", n);

    if (!be->writeFile(argv[2])) { fprintf(stderr, "writeFile failed\n"); return 1; }
    return 0;
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.14)
project(ifmc_repro CXX)
find_package(Dyninst REQUIRED COMPONENTS dyninstAPI)
add_executable(mutator mutator.cpp)
target_link_libraries(mutator Dyninst::dyninstAPI)
target_compile_features(mutator PRIVATE cxx_std_17)

Build & run

# mutatee
gcc -O0 -g mutatee.c -o mutatee

# mutator (against an installed Dyninst)
cmake -S . -B build -DDyninst_DIR=<dyninst-install>/lib/cmake/Dyninst
cmake --build build

export DYNINSTAPI_RT_LIB=$(ls <dyninst-install>/lib*/libdyninstAPI_RT.so.* | head -1)
export LD_LIBRARY_PATH=<dyninst-install>/lib64:$LD_LIBRARY_PATH   # + boost/tbb/elfutils dirs

# control: bare snippet, no ifMachineCondition
./build/mutator mutatee mutatee.plain plain && ./mutatee.plain; echo "exit=$?"
# bug: ifMachineCondition
./build/mutator mutatee mutatee.cond        && ./mutatee.cond;  echo "exit=$?"

Expected vs. actual

Mode	Correct behavior	Actual (current Dyninst)
plain (control)	runs; access_count = 11 (unconditional)	✅ runs, exit 0
cond (ifMachineCondition)	runs; access_count = 5 (only when x > 0)	❌ crashes — exit=135 (SIGBUS)

The control proves the point is instrumentable and the program is otherwise
sound; adding only the ifMachineCondition wrapper — the sole change that emits
emitRestoreFlagsFromStackSlot — crashes it.

Reachability

Both are documented public entry points:

• BPatch_ifMachineConditionExpr → operatorAST ifMCOp → emitJmpMC
  (dyninstAPI/src/ASTs/operatorAST.C) → emitRestoreFlagsFromStackSlot.
• BPatch_effectiveAddressExpr / BPatch_bytesAccessedExpr on a rep scas/cmps
  → BPatch_memoryAccessAdapter → emitCSload(..., IA32_ESCAS/IA32_ECMPS) →
  same emitRestoreFlagsFromStackSlot.

Suggested fix

Make producer and consumer agree on a real RFLAGS image — either:

• emit a real pushfq in saveVolatileRegisters (and wire up or delete the dead
  emitPushFlags), so the consumer's popfq is correct (DF included); or
• have emitRestoreFlagsFromStackSlot reconstruct the flags from the
  lahf/seto %rax form (add $0x7f,%al; sahf) instead of popfq.

The first also fixes DF, which the current lahf/seto sequence can never
preserve.

Environment

• x86-64, gcc 15.2, Dyninst built from master, binary rewriter (openBinary).
• Reproduced deterministically; crash signal is SIGBUS on this host (AC), and can
  present as SIGTRAP (TF) or a wrong branch depending on the live flags at the point.