Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

max_temp_directory_size in connect config is reported but not enforced

Aperta
  • Qualcosa non va?
#667 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
30/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
databases

Direzione di ricerca

Run the reproduction script first to confirm the two runs differ: the apply_set=False run should succeed while the apply_set=True run raises OutOfMemoryException. Then trace how the max_temp_directory_size key in the config= mapping passed to duckdb.connect is applied during connection initialization, and compare that with the path SET max_temp_directory_size takes. Done means the connect-time 1-byte cap rejects the spill just as the SET run does, and the same script passes both cases.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happens?

Passing max_temp_directory_size through duckdb.connect(config=...) makes current_setting('max_temp_directory_size') report the requested value, but spilling is not bounded by it. Executing the same value with SET max_temp_directory_size after the connection is created enforces the cap.

This can silently let a Python process consume the filesystem space that the connect-time configuration was intended to reserve.

To Reproduce

This self-contained script generates its own Parquet input and runs the same external sort twice:

import tempfile
from pathlib import Path

import duckdb


def run(apply_set: bool) -> None:
    root = Path(tempfile.mkdtemp())
    source = root / "source.parquet"
    output = root / "output.parquet"

    with duckdb.connect() as connection:
        connection.sql(
            """
            SELECT
                i AS row_id,
                i % 17 AS group_id,
                repeat(md5(i::VARCHAR), 32) AS payload
            FROM range(49152) t(i)
            """
        ).write_parquet(str(source), row_group_size=2048)

    with duckdb.connect(
        str(root / "test.duckdb"),
        config={
            "memory_limit": "64MB",
            "threads": 2,
            "temp_directory": str(root / "tmp"),
            "preserve_insertion_order": False,
            "max_temp_directory_size": "1B",
        },
    ) as connection:
        if apply_set:
            connection.execute("SET max_temp_directory_size = '1B'")

        print("apply_set", apply_set)
        print(
            "setting",
            connection.execute(
                "SELECT current_setting('max_temp_directory_size')"
            ).fetchone(),
        )

        try:
            connection.read_parquet(str(source)).order(
                "group_id, payload, row_id"
            ).write_parquet(str(output), row_group_size=2048)
            print("sort succeeded")
        except Exception as error:
            print(type(error).__name__, str(error).splitlines()[0])


run(apply_set=False)
run(apply_set=True)

Output with DuckDB 1.5.6:

apply_set False
setting ('1 byte',)
sort succeeded
apply_set True
setting ('1 byte',)
OutOfMemoryException Out of Memory Error: failed to offload data block of size 32.0 KiB (0 bytes/1 byte used).

Expected: both runs reject the spill at the 1-byte cap.

The behavior also reproduces with DuckDB 1.5.2. This is specific to initialization through the Python config mapping, so there is no equivalent plain-SQL/CLI connect-time reproduction; the post-connect SQL SET is the working comparison.

Environment
  • OS: macOS 27.0, arm64
  • DuckDB Package Version: 1.5.6 (latest stable; also 1.5.2)
  • Python Version: 3.13.14
  • Full Name: Simon Thelin
  • Affiliation: Odyssey
  • Latest build tested: stable release
  • Data set: not applicable; the reproduction generates its input
  • All code and relevant configuration are included
Lingua principale
Python
Stelle
191
Fork
120
Merge medio
14h 1m
PR unite (30g)
23

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di duckdb/duckdb-python

Tutte le issue di duckdb/duckdb-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.