feat(tools/terminal): add hard_timeout mode to terminate timed-out command processes and surface actionable timeout errors

Open
#7,017 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@zsicheng0214 is already working on this.

Since Nov 20, 2026.

  • #7025 by @zsicheng0214 — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
cli, tooling

Research direction

Start with TerminalSession._handle_hard_timeout_command in openhands-tools/openhands/tools/terminal/terminal/terminal_session.py, then trace TerminalAction and TerminalObservation through the tool schema and serialization. Read the timeout tests and Unix and Windows terminal tool descriptions to find the relevant entry points. Done when hard timeouts terminate the command, return the specified error fields and suffix, and existing soft-timeout behavior remains unchanged.

Written by the indexing model from the issue text.

Description

priority:low ready-for-dev

Motivation

In TerminalTool, TerminalAction.timeout currently acts strictly as a soft observation pause: when action.timeout expires (TerminalSession._handle_hard_timeout_command in openhands-tools/openhands/tools/terminal/terminal/terminal_session.py), the session captures the screen buffer and returns exit_code=-1, is_error=False while leaving the child process running in the background inside the terminal / pooled tmux pane until a future independent command happens to reuse that same pane.

While soft pause is useful for interactive commands where the agent wants to inspect progress or send C-c, automated pipelines, test runners, firmware flashes, and bounded-deadline probes frequently require a strict hard deadline where:

  1. A runaway command must not continue mutating workspace files in the background after its deadline expires (especially in pooled tmux mode where the next command may check out a different pane while the timed-out pane keeps writing);
  2. Post-tool observers, hooks, and agents need is_error=True, timeout=True, and an explicit machine-readable process_terminated=True indicator distinguishing a terminated hard deadline from an interactive soft pause.

In containerized TraceTrial-Bench evaluation across 95 tasks, this gap directly caused failures across both E3 (timeout boundary & orphan process convergence: E3-firmware-timeout-001, E3-longrun-timeout-002, E3-pipeline-timeout-003) and E6 (E6-timeout-error-channel-003).

Proposed Solution

Keep hard_timeout: bool = False as the default (preserving 100% backward compatibility for interactive soft-timeout workflows) and add opt-in strict deadline enforcement:

  • Add hard_timeout: bool = False to TerminalAction.
  • Add process_terminated: bool = False to TerminalObservation.
  • When action.hard_timeout=True and action.timeout expires before completion:
    1. Immediately interrupt the running command (terminal.interrupt()) and wait boundedly for prompt recovery (closing/marking the session for automatic self-healing if the process ignores SIGINT);
    2. Reset prev_status cleanly when the prompt recovers so subsequent commands start from a clean prompt;
    3. Return TerminalObservation with timeout=True, is_error=True, process_terminated=True, and a clear [The command timed out after X seconds and was terminated.] suffix.

OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

Verified against main: TerminalSession._handle_hard_timeout_command (openhands-tools/openhands/tools/terminal/terminal/terminal_session.py) returns exit_code=-1 with neither timeout=True nor is_error=True and leaves the child process running; cleanup happens only later at the next pooled-pane checkout (TerminalExecutor._prepare_pooled_session interrupts and polls at most ~2s). The problem statement is accurate.

Bounded scope:

  • Add hard_timeout: bool = False to TerminalAction and process_terminated: bool = False to TerminalObservation — both additive with False defaults, preserving backward compatibility.
  • Enforce termination only when hard_timeout=True and action.timeout expires: interrupt via the existing TerminalInterface.interrupt() (implemented on subprocess, tmux, and Windows backends), bound the prompt-recovery wait, and fall back to the repository's existing self-healing paths (session close with SIGTERM→SIGKILL escalation in subprocess_terminal.py, TmuxPanePool.replace()) when the prompt does not recover; report timeout=True, is_error=True, process_terminated=True, and a terminated-timeout suffix.
  • Trace the new fields through the tool schema/serialization surface and document the mode in the terminal tool descriptions (Unix and Windows).

Non-goals:

  • No change to default soft-timeout behavior, the no-change-timeout path, or max_observation_wait_seconds handling.
  • No new process-kill API on TerminalInterface beyond the existing interrupt() and close/replace self-healing; no PID-tracking surface.
  • No changes to other tools; does not address #6651 (pool Ctrl+C delivery to a still-running command).
Acceptance Criteria
  • With hard_timeout=True and timeout=N on a command still running at the deadline, the returned TerminalObservation has timeout=True, is_error=True, process_terminated=True, exit_code=-1, and its text includes a suffix stating the command timed out after N seconds and was terminated.
  • The observation includes the output the command produced before the deadline (subject to the existing MAX_CMD_OUTPUT_SIZE truncation).
  • After the observation returns, the timed-out process is no longer running on that pane/session (the backend reports no running command and a recovered prompt), so it cannot keep mutating workspace files in the background.
  • When the timed-out process ignores SIGINT, the tool still returns the timeout error observation and the session recovers via the existing close/replace self-healing path within a bounded wait, so the next command runs from a clean prompt instead of hanging.
  • In pooled tmux mode, an independent follow-up command on the same executor after a hard timeout executes as a new command (not as a follow-up interaction with the timed-out command), with session prompt/prev_status state clean.
  • Default behavior is unchanged: actions with hard_timeout=False (or the field absent) keep the current soft-timeout result — no interrupt or termination, is_error=False, process_terminated=False, existing timeout suffix — and existing serialized actions/observations without the new fields still parse with the defaults.
  • hard_timeout=True without a timeout deadline is inert: no termination occurs and behavior matches the current soft flow.
  • hard_timeout=True does not bypass the foreground timeout cap: when OH_RUNTIME_IDLE_TIMEOUT_SECONDS is configured and the requested timeout exceeds the cap, the existing foreground-timeout rejection observation is returned before execution.
  • The new fields appear in the terminal tool schema with defaults of False, and the Unix and Windows tool descriptions document the hard-timeout mode's terminate-on-deadline semantics versus the soft pause.
  • Automated tests against real sessions (tmux and subprocess backends) demonstrate that a long-running command with hard_timeout=True returns the terminated-error observation and its process is gone afterwards; the existing timeout tests pass unchanged.
Dominant language
Python
Stars
1.4k
Forks
797
Avg merge
1d 17h
Merged PRs (30d)
354

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OpenHands/software-agent-sdk

All issues in OpenHands/software-agent-sdk

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.