DuckDB spawns worker threads on module import rather than connection creation

Start by reproducing the issue with import duckdb on the reported Ubuntu setup and inspect the import path and duckdb.connect() initialization. Use the provided gdb backtrace, especially ExecuteForever, to trace when the worker pool is created. Done means importing the module alone does not spawn the worker threads, while creating a connection still initializes the required workers.

What happens?

Importing the duckdb Python module (import duckdb) immediately spawns ~100 worker threads (appears to be ncpus), even without creating any connections. This causes unexpected resource consumption in applications that import the module but may not immediately use it.

Expected Behavior

Worker threads should only be created when:

1. An actual database connection is established via duckdb.connect()
2. Or when explicitly configured to do so

The module import itself should be lightweight and not spawn background threads.

Proposed Solution

Consider lazy initialization of the thread pool:

• Defer TaskScheduler/thread pool creation until first connect() call
• Or provide an environment variable/config option to control thread spawning behavior on import
• Similar to how other database drivers handle connection pooling

Workaround

Currently requires monkey-patching or lazy imports to avoid the thread spawning:

# Must be done before any code imports duckdb
import sys
import importlib.util

def lazy_import_duckdb():
    spec = importlib.util.find_spec("duckdb")
    module = importlib.util.module_from_spec(spec)
    sys.modules["duckdb"] = module
    # Don't execute the module yet
    return module

To Reproduce

import duckdb  # This alone spawns ~100 threads

When attaching gdb to a process that has imported duckdb:

• thread apply all bt shows 100+ threads with duckdb in their stack traces
• All threads show ExecuteForever in their call stacks
• Base stack frame is clone3 syscall

OS:

Ubuntu 22.04 x86_64

DuckDB Version:

0.6.1

DuckDB Client:

Python

Hardware:

128 core CPU

Full Name:

Nova DasSarma

Affiliation:

Anthropic, PBC

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have