Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

max_temp_directory_size in connect config is reported but not enforced

Abierto
  • ¿Algo no está bien?
#667 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
databases

Línea de trabajo

Run the reproduction script first to confirm the two runs differ: the apply_set=False run should succeed while the apply_set=True run raises OutOfMemoryException. Then trace how the max_temp_directory_size key in the config= mapping passed to duckdb.connect is applied during connection initialization, and compare that with the path SET max_temp_directory_size takes. Done means the connect-time 1-byte cap rejects the spill just as the SET run does, and the same script passes both cases.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

What happens?

Passing max_temp_directory_size through duckdb.connect(config=...) makes current_setting('max_temp_directory_size') report the requested value, but spilling is not bounded by it. Executing the same value with SET max_temp_directory_size after the connection is created enforces the cap.

This can silently let a Python process consume the filesystem space that the connect-time configuration was intended to reserve.

To Reproduce

This self-contained script generates its own Parquet input and runs the same external sort twice:

import tempfile
from pathlib import Path

import duckdb


def run(apply_set: bool) -> None:
    root = Path(tempfile.mkdtemp())
    source = root / "source.parquet"
    output = root / "output.parquet"

    with duckdb.connect() as connection:
        connection.sql(
            """
            SELECT
                i AS row_id,
                i % 17 AS group_id,
                repeat(md5(i::VARCHAR), 32) AS payload
            FROM range(49152) t(i)
            """
        ).write_parquet(str(source), row_group_size=2048)

    with duckdb.connect(
        str(root / "test.duckdb"),
        config={
            "memory_limit": "64MB",
            "threads": 2,
            "temp_directory": str(root / "tmp"),
            "preserve_insertion_order": False,
            "max_temp_directory_size": "1B",
        },
    ) as connection:
        if apply_set:
            connection.execute("SET max_temp_directory_size = '1B'")

        print("apply_set", apply_set)
        print(
            "setting",
            connection.execute(
                "SELECT current_setting('max_temp_directory_size')"
            ).fetchone(),
        )

        try:
            connection.read_parquet(str(source)).order(
                "group_id, payload, row_id"
            ).write_parquet(str(output), row_group_size=2048)
            print("sort succeeded")
        except Exception as error:
            print(type(error).__name__, str(error).splitlines()[0])


run(apply_set=False)
run(apply_set=True)

Output with DuckDB 1.5.6:

apply_set False
setting ('1 byte',)
sort succeeded
apply_set True
setting ('1 byte',)
OutOfMemoryException Out of Memory Error: failed to offload data block of size 32.0 KiB (0 bytes/1 byte used).

Expected: both runs reject the spill at the 1-byte cap.

The behavior also reproduces with DuckDB 1.5.2. This is specific to initialization through the Python config mapping, so there is no equivalent plain-SQL/CLI connect-time reproduction; the post-connect SQL SET is the working comparison.

Environment
  • OS: macOS 27.0, arm64
  • DuckDB Package Version: 1.5.6 (latest stable; also 1.5.2)
  • Python Version: 3.13.14
  • Full Name: Simon Thelin
  • Affiliation: Odyssey
  • Latest build tested: stable release
  • Data set: not applicable; the reproduction generates its input
  • All code and relevant configuration are included
Lenguaje dominante
Python
Estrellas
191
Forks
120
Merge medio
1 d 20 min
PR fusionados (30 d)
41

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de duckdb/duckdb-python

Todos los issues de duckdb/duckdb-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.