Windows native gem: SQLite's Sleep() resolves to Ruby's rb_w32_Sleep, releasing the GVL (and delivering interrupts) while SQLite holds the connection mutex

Read the Windows native extension initialization path and its SQLite VFS setup, then inspect the vendored SQLite os_win.c system call table and xSetSystemCall support. Confirm how the Windows gem binds Sleep and whether initialization occurs after sqlite3_initialize() but before connections open. Done means Windows SQLite sleeps use KERNEL32!Sleep and a Windows native gem check confirms the regression is fixed.

Summary

On Windows, the precompiled x64-mingw-ucrt gem's sqlite3_native.so imports Sleep from the Ruby DLL (x64-ucrt-ruby400.dll), not from KERNEL32.dll. Ruby exports Sleep as a forwarder to rb_w32_Sleep, which sleeps inside BLOCKING_REGION: it releases the GVL and runs Ruby's interrupt check.

Every sleep SQLite takes inside a call therefore releases the GVL while SQLite still holds the connection's mutex (db->mutex), and can deliver a pending Thread#raise/#kill inside SQLite's C frames. That covers the default busy handler behind busy_timeout, winLock's pending-lock retry, and winRetryIoerr.

That causes two failures:

1. A whole-process deadlock when two Ruby threads use one SQLite3::Database. Thread A sleeps inside SQLite with the connection mutex held and the GVL released. Thread B calls into the same connection and blocks in EnterCriticalSection holding the GVL. A can never get the GVL back. The process stops for good at 0% CPU.
2. A connection mutex left locked by an interrupt. rb_w32_Sleep → BLOCKING_REGION → blocking_region_begin → unblock_function_set → RUBY_VM_CHECK_INTS, so a Thread#raise, Thread#kill or Timeout pending at that point is delivered right there. It unwinds through SQLite's frames with db->mutex locked, and the next call on that connection blocks forever with the GVL held.

Neither can happen on Linux or macOS, where SQLite's sleeps go to libc and keep the GVL. With the same Ruby code, the Windows gem behaves differently on this point from the other platforms, and without any warning.

We hit (1) in production (details below). I think the cleanest fix is in the gem: route SQLite's Sleep back to KERNEL32!Sleep.

Environment

• Windows 10/11 x64, RubyInstaller-based Ruby 4.0.9 (x64-mingw-ucrt).
• sqlite3-2.10.1-x64-mingw-ucrt (vendored SQLite 3.54.1). The same import appears in the x64-mingw-ucrt builds of 2.9.6, 2.10.1 and 2.10.2, which are all the versions I checked.

Evidence

1. The gem imports Sleep from the Ruby DLL

x86_64-w64-mingw32-objdump -p lib/sqlite3/4.0/sqlite3_native.so (2.10.1, and likewise 2.9.6 and 2.10.2):

DLL Name: x64-ucrt-ruby400.dll
    ...
    Sleep
    getpid
    ...

These are the only non-rb_*/ruby_* names imported from the Ruby DLL. getpid maps to rb_w32_getpid and is harmless.

2. Why the Ruby DLL has a Sleep export

win32/mkexports.rb reads include/ruby/win32.h. For each #define X(args) rb_w32_X(args) substitution it adds an export named X that forwards to rb_w32_X:

// include/ruby/win32.h
#define Sleep(msec)		(void)rb_w32_Sleep(msec)

so x64-ucrt-ruby400.dll exports both Sleep and rb_w32_Sleep, at the same address. The extension links the static libsqlite3.a that extconf.rb builds through mini_portile, and links Ruby's import library ahead of kernel32. SQLite's reference to Sleep (its aSyscall[] table in os_win.c: { "Sleep", (SYSCALL)Sleep, 0 }) is then resolved against Ruby's import library.

I confirmed the binding in a full-memory dump of a running process: SQLite's Sleep syscall slot → import thunk jmp *[IAT] → IAT entry = x64-ucrt-ruby400.dll!rb_w32_Sleep.

3. What rb_w32_Sleep does (Ruby 4.0.9, thread_win32.c)

int WINAPI
rb_w32_Sleep(unsigned long msec)
{
    int ret;
    rb_thread_t *th = GET_THREAD();

    BLOCKING_REGION(th, ret = rb_w32_sleep(msec),
                    ubf_handle, ruby_thread_from_native(), FALSE);
    return ret;
}

It releases the GVL around the sleep. With fail_if_interrupted == FALSE, blocking_region_begin → unblock_function_set runs RUBY_VM_CHECK_INTS(th->ec) (thread.c), which delivers pending interrupts.

4. Where SQLite sleeps while holding db->mutex (os_win.c, SQLite 3.54.1)

All of these go through osSleep, the aSyscall[] slot for Sleep:

• winLock(): when taking a SHARED lock (or going from RESERVED to EXCLUSIVE), it tries LockFileEx on the PENDING byte up to 3 times with osSleep(1) between tries ("Try 3 times to get the pending lock. This is needed to work around problems caused by indexing and/or anti-virus software"). This happens whenever another process holds that byte, and it does not depend on the busy handler: it happens with busy_timeout(0) too.
• winRetryIoerr(): osSleep(winIoerrRetryDelay*(1+n)) on sharing or lock violations.
• The default busy callback that Database#busy_timeout= installs: sqliteDefaultBusyCallback → sqlite3OsSleep → winSleep → osSleep.

Real-world occurrence

Lich (a MUD client proxy) runs several Ruby processes against one SQLite file, and Windows users running several sessions saw processes freeze permanently. The write-up with native stacks, Ruby backtraces and mutex owners is here: https://github.com/elanthia-online/lich-5/issues/2254#issuecomment-6647684262. In short, from a full-memory dump of a frozen process:

• Ruby's GVL (a Win32 mutant) was owned by thread A, which was blocked in RtlEnterCriticalSection ← sqlite3_native.so ← Database#exec_batch (execute_batch2('PRAGMA data_version')).
• The critical section (an sqlite3_mutex with id == SQLITE_MUTEX_RECURSIVE, i.e. a connection's db->mutex) was owned by thread B. B was in execute_batch2 on the same connection, inside SQLite's winLock retry loop, inside rb_w32_Sleep, waiting for the GVL.
• About 20 other threads were queued on the GVL.

We have fixed our side (lich-5 PR #2268): calls on that shared connection are now serialized, and interrupts are deferred around them. But any application that shares a connection between threads on Windows can hit this, and the interrupt variant (2) can hit even single-thread users of a connection that runs Thread#raise/Timeout code.

Reproduction

Caveat: I could not run these on Windows myself. I ran both on Linux (Ruby 4.0.9, sqlite3 2.10.2): (a) prints ticks during C busy wait: 0, and (b) survives 30 s. The Windows results below are predictions from the analysis above; I'd be grateful if someone with a Windows box can confirm.

(a) The GVL is released during a C busy-timeout wait. This is the quickest signal: it should tick on Windows and show about 0 ticks on Linux and macOS.

require "sqlite3"
require "tmpdir"
require "rbconfig"

path = File.join(Dir.mktmpdir, "t.db")
SQLite3::Database.new(path) { |d| d.execute("create table t(x)") }

# Another process holds the write lock for 3 s.
holder = spawn(RbConfig.ruby, "-rsqlite3", "-e", <<~R, path)
  d = SQLite3::Database.new(ARGV[0])
  d.execute("begin exclusive"); sleep 3; d.execute("commit")
R
sleep 1

db = SQLite3::Database.new(path)
db.busy_timeout = 1500 # C busy handler; on Linux this wait holds the GVL
ticks = 0
ticker = Thread.new { loop { sleep 0.01; ticks += 1 } }
begin
  db.execute("insert into t values (1)")
rescue SQLite3::BusyException
end
puts "ticks during C busy wait: #{ticks}" # Linux/macOS: ~0. Windows (prediction): ~100+
Process.wait(holder)

(b) Deadlock with one connection shared by two threads. This is a stress test, so it may take a few seconds to hit. When it deadlocks, the heartbeat lines stop and the process sits at 0% CPU until killed.

require "sqlite3"
require "tmpdir"
require "rbconfig"

path = File.join(Dir.mktmpdir, "t.db")
SQLite3::Database.new(path) { |d| d.execute("create table t(x)") }

# Another process takes and releases the exclusive lock in a loop, so winLock
# in this process keeps hitting its PENDING-lock retry (osSleep(1)).
holder = spawn(RbConfig.ruby, "-rsqlite3", "-e", <<~R, path)
  d = SQLite3::Database.new(ARGV[0])
  d.busy_timeout = 50
  loop do
    d.execute("begin exclusive")
    d.execute("insert into t values (1)")
    d.execute("commit")
  rescue SQLite3::BusyException
    nil
  end
R

db = SQLite3::Database.new(path) # one connection, two threads, no busy timeout
2.times do
  Thread.new do
    loop do
      db.execute_batch2("PRAGMA data_version")
    rescue SQLite3::BusyException
      nil
    end
  end
end

30.times { |i| sleep 1; puts "heartbeat #{i}" }
puts "survived 30 s (no deadlock this run)"
Process.kill(:KILL, holder)

(c) The interrupt variant: as in (a), but call Thread#raise on the thread waiting in execute, then use db from another thread. On Windows I'd expect the second call to hang with the GVL held. On Linux the interrupt can't land inside SQLite, because the wait holds the GVL.

Suggested fix

Make SQLite's sleeps on Windows call KERNEL32!Sleep again, the same as SQLite's sleeps on every other platform.

Option 1 (runtime, no dependence on link order; I'd suggest this one): in Init_sqlite3_native, on Windows, point SQLite's Sleep system call at kernel32's through the VFS's xSetSystemCall. os_win.c supports this through winSetSystemCall, which looks the entry up by name in aSyscall[]:

#ifdef _WIN32
static void
rb_sqlite3_use_kernel32_sleep(void)
{
    HMODULE k32 = GetModuleHandleW(L"kernel32.dll");
    sqlite3_syscall_ptr real_sleep = k32 ? (sqlite3_syscall_ptr)GetProcAddress(k32, "Sleep") : NULL;
    const char *names[] = { "win32", "win32-longpath", "win32-none", "win32-longpath-none" };
    if (!real_sleep) return;
    for (size_t i = 0; i < sizeof(names) / sizeof(names[0]); i++) {
        sqlite3_vfs *vfs = sqlite3_vfs_find(names[i]);
        if (vfs && vfs->iVersion >= 3 && vfs->xSetSystemCall) {
            vfs->xSetSystemCall(vfs, "Sleep", real_sleep);
        }
    }
}
#endif

aSyscall[] is shared by all the Windows VFSes, so a single call is probably enough; looping over the names is defensive. This has to run after sqlite3_initialize() and before any connection opens. (I've only compile-checked this sketch, with x86_64-w64-mingw32-gcc -Wall -Wextra against the vendored sqlite3.h; I haven't run it.) It also fixes users who link a system SQLite built the same way.

Option 2 (build time): link -lkernel32 ahead of Ruby's import library for the extension, so __imp_Sleep resolves to kernel32. That is fragile: it depends on mkmf's link-order details, and any other #define X(...) rb_w32_X(...) name SQLite might pick up in the future (win32.h has many) would bring the problem back.

Either way, a check like reproduction (a) in CI on the Windows native gem would guard against a regression. Checking that objdump -p / dumpbin /imports shows no Sleep import from the Ruby DLL would also work.

Behaviour change to note: after the fix, busy_timeout waits on Windows hold the GVL, as they already do on Linux and macOS. Users who want lock waits that let other threads run already have Database#busy_handler_timeout=, which waits in Ruby. That's arguably the documented behaviour anyway; today's Windows behaviour (GVL released while SQLite's mutex is held, interrupts delivered mid-call) is the unsafe one.

Notes

• I have not checked whether a from-source install (force_ruby_platform) on Windows links the same way, but I'd expect it to, since it uses the same extconf and link order.
• Other C extensions that link a static library calling Sleep probably get the same binding; a note to Ruby core about the forwarding exports may be worthwhile. Fixing it in this gem seems most direct, though, since SQLite holds its own mutexes across these sleeps.

Thanks for maintaining this gem, and for the 2.10.2 callback and interrupt fixes, which closed a related hazard on our side.

🤖 Generated with Claude Code