Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

max_temp_directory_size in connect config is reported but not enforced

Aberta
  • Algo errado?
#667 1 comentário 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
30/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
python
Domínio
databases

Direção de pesquisa

Run the reproduction script first to confirm the two runs differ: the apply_set=False run should succeed while the apply_set=True run raises OutOfMemoryException. Then trace how the max_temp_directory_size key in the config= mapping passed to duckdb.connect is applied during connection initialization, and compare that with the path SET max_temp_directory_size takes. Done means the connect-time 1-byte cap rejects the spill just as the SET run does, and the same script passes both cases.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

What happens?

Passing max_temp_directory_size through duckdb.connect(config=...) makes current_setting('max_temp_directory_size') report the requested value, but spilling is not bounded by it. Executing the same value with SET max_temp_directory_size after the connection is created enforces the cap.

This can silently let a Python process consume the filesystem space that the connect-time configuration was intended to reserve.

To Reproduce

This self-contained script generates its own Parquet input and runs the same external sort twice:

import tempfile
from pathlib import Path

import duckdb


def run(apply_set: bool) -> None:
    root = Path(tempfile.mkdtemp())
    source = root / "source.parquet"
    output = root / "output.parquet"

    with duckdb.connect() as connection:
        connection.sql(
            """
            SELECT
                i AS row_id,
                i % 17 AS group_id,
                repeat(md5(i::VARCHAR), 32) AS payload
            FROM range(49152) t(i)
            """
        ).write_parquet(str(source), row_group_size=2048)

    with duckdb.connect(
        str(root / "test.duckdb"),
        config={
            "memory_limit": "64MB",
            "threads": 2,
            "temp_directory": str(root / "tmp"),
            "preserve_insertion_order": False,
            "max_temp_directory_size": "1B",
        },
    ) as connection:
        if apply_set:
            connection.execute("SET max_temp_directory_size = '1B'")

        print("apply_set", apply_set)
        print(
            "setting",
            connection.execute(
                "SELECT current_setting('max_temp_directory_size')"
            ).fetchone(),
        )

        try:
            connection.read_parquet(str(source)).order(
                "group_id, payload, row_id"
            ).write_parquet(str(output), row_group_size=2048)
            print("sort succeeded")
        except Exception as error:
            print(type(error).__name__, str(error).splitlines()[0])


run(apply_set=False)
run(apply_set=True)

Output with DuckDB 1.5.6:

apply_set False
setting ('1 byte',)
sort succeeded
apply_set True
setting ('1 byte',)
OutOfMemoryException Out of Memory Error: failed to offload data block of size 32.0 KiB (0 bytes/1 byte used).

Expected: both runs reject the spill at the 1-byte cap.

The behavior also reproduces with DuckDB 1.5.2. This is specific to initialization through the Python config mapping, so there is no equivalent plain-SQL/CLI connect-time reproduction; the post-connect SQL SET is the working comparison.

Environment
  • OS: macOS 27.0, arm64
  • DuckDB Package Version: 1.5.6 (latest stable; also 1.5.2)
  • Python Version: 3.13.14
  • Full Name: Simon Thelin
  • Affiliation: Odyssey
  • Latest build tested: stable release
  • Data set: not applicable; the reproduction generates its input
  • All code and relevant configuration are included
Linguagem predominante
Python
Estrelas
191
Forks
120
Merge médio
14h 1min
PRs com merge (30d)
23

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de duckdb/duckdb-python

Todas as issues de duckdb/duckdb-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.