General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`

Comece com python/tools/python3src.zip/semmle/logging.py:84 e semmle/worker.py:115-116 e, em seguida, reproduza a falha com o comando sandboxed multiprocessing Queue e codeql database create. Rastreie os caminhos de inicialização de Queue e de processos e verifique se a criação do banco de dados Python é concluída quando os semáforos POSIX são negados, incluindo os caminhos de logger e extractor pool.

Summary

The Python extractor unconditionally uses multiprocessing.Queue and multiprocessing.Process, which require POSIX semaphores (sem_open()). In macOS App Sandbox environments (Seatbelt), ipc-posix-sem is denied at the kernel level, making codeql database create --language=python impossible.

No existing issue covers this — searched for SemLock, semaphore, PermissionError macos, sandbox, multiprocessing with zero matches.

Environment

• CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
• Python extractor version: 7.1.8
• macOS: Darwin 24.6.0 (Sequoia, arm64)
• Python: 3.12 and 3.14 (both fail identically)
• Sandbox: macOS Seatbelt (sandbox-exec) — used by Claude Code, Codex, and other sandboxed developer tools

Reproduction

Run codeql database create inside any macOS App Sandbox that denies ipc-posix-sem:

# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted

# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite

Fails at:

File ".../python3src.zip/semmle/logging.py", line 85, in __init__
    self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted

If the logger is patched to bypass this, a second identical failure occurs in semmle/worker.py:115 (ExtractorPool.__init__ → ctx.Queue(proc_count*2)).

Affected Code

1. python/tools/python3src.zip → semmle/logging.py:84 — Logger.__init__ unconditionally creates multiprocessing.Queue() and spawns a Process for log message routing, regardless of verbosity level.

2. python/tools/python3src.zip → semmle/worker.py:115-116 — ExtractorPool.__init__ creates multiprocessing.Queue and multiprocessing.Process workers for parallel extraction.

Both use multiprocessing.get_context('spawn') on macOS, which calls sem_open().

Why This Matters

macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The ipc-posix-sem denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.

Suggested Fix

Add a fallback to threading.Thread + queue.Queue when multiprocessing is unavailable or fails. This is the same pattern used for AWS Lambda (where /dev/shm is unavailable) and Docker containers with restricted IPC namespaces.

A minimal change: catch PermissionError/OSError in Logger.__init__ and ExtractorPool.__init__, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).