Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Streaming jobs can bypass timeouts when output has no newline

Cerrado
  • ¿Algo no está bien?
#9,528 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 5 días

Ya se ha fusionado un pull request relacionado.

  • #9529 de @MateehUllah — fusionado

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
65/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend, cli, tooling

Línea de trabajo

The issue is in packages/syft-job/src/syft_job/job_runner.py in the _execute_job_streaming() function. Start by understanding the selector loop and the readline() call. Look at how timeouts are currently checked. The fix likely involves reading available bytes without blocking, perhaps using os.read or checking fileobj.read(1) in a non-blocking mode. Write a test that reproduces the bug using a script that outputs without a newline and sleeps. Verify the timeout is enforced after your changes.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Type: Bug :bug:

Description

The streaming job runner can block while reading output that does not end with a newline, preventing the configured timeout from being enforced.

In packages/syft-job/src/syft_job/job_runner.py, _execute_job_streaming() waits for readable output using a selector and then calls:

line = key.fileobj.readline()

A pipe becoming readable only guarantees that some data is available. It does not guarantee that a complete line is available. If a job writes partial output without a newline and then continues running, readline() can block until the process writes a newline or exits.

While blocked inside readline(), the loop cannot execute its timeout check. If the process exits before readline() returns, the job may finish without being marked as timed out even though it exceeded the configured timeout.

How to Reproduce

  1. Create a job whose run.sh writes output without a trailing newline and then sleeps:

    #!/bin/bash
printf "working"
sleep 10

  2. Run the job using streaming output with a timeout shorter than the sleep duration, for example:

    timeout=1

  3. Observe that the selector detects the available working output.

  4. The subsequent readline() call waits for the process to exit because no newline was written.

  5. The one-second timeout is not enforced while the runner is blocked.

A minimal Python reproduction of the underlying behavior is:

import selectors
import subprocess
import time

process = subprocess.Popen(
    ["bash", "-c", "printf working; sleep 5"],
    stdout=subprocess.PIPE,
    text=True,
)

selector = selectors.DefaultSelector()
selector.register(process.stdout, selectors.EVENT_READ)

start = time.monotonic()

for key, _ in selector.select(timeout=0.1):
    output = key.fileobj.readline()
    print(output, time.monotonic() - start)

Although output becomes available immediately, readline() returns only after approximately five seconds when the process exits.

Expected Behavior

The streaming runner should enforce the configured timeout regardless of whether job output contains newline characters.

Reading available output should not block the monitoring loop. Possible approaches include non-blocking reads, reading available chunks instead of complete lines, or otherwise ensuring that timeout checks continue while partial output is buffered.

Tests should cover a process that writes partial output without a newline and then runs longer than its configured timeout.

Screenshots

Not applicable. This behavior occurs in the job runner’s process-monitoring logic.

System Information

  • OS: Reproduced on a POSIX environment
  • OS Version: Not applicable
  • Language Version: Python
  • Package Manager Version: Not applicable
  • Browser: Not applicable
  • Browser Version: Not applicable

Additional Context

The non-streaming execution path uses process.communicate(timeout=timeout) and is not affected by this specific blocking readline() behavior.

Duplicate searches for streaming output without newlines, selector/readline timeout behavior, and hanging job timeouts found no matching open or closed issue.

Lenguaje dominante
Python
Estrellas
10k
Forks
2k
Merge medio
4 d 15 h
PR fusionados (30 d)
29

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de OpenMined/PySyft

Todos los issues de OpenMined/PySyft

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.