`test_control_mode_stdout_preserves_non_ascii_output` polls `select` on a buffered stream

Bắt đầu trong tests/test_control_mode.py tại test_control_mode_stdout_preserves_non_ascii_output và kiểm tra vòng lặp select/readline. Thay thế việc chờ dựa trên select bằng reader thread queue.Queue và deadline được đề xuất, sau đó chạy test tập trung, bao gồm cả khi đã loại bỏ hồi quy mã hóa UTF-8, để xác nhận rằng phạm vi bao phủ UnicodeDecodeError vẫn được duy trì.

Filed against tmux-python/libtmux v0.62.0, tmux 3.7b. Flaked twice in CI on unrelated pull requests, on different tmux matrix cells, both times passing on re-run.

What happens

test_control_mode_stdout_preserves_non_ascii_output bounds its wait with select.select() on a buffered stream and then consumes with readline(), so readiness and readability are measured against two different buffers.

tests/test_control_mode.py#L88-L94

ControlMode builds its subprocess with text=True, encoding="utf-8", so stdout is a TextIOWrapper over a BufferedReader:

src/libtmux/_internal/control_mode.py#L83-L90 and #L99

select calls fileno() and sees only the kernel pipe. One readline() drains up to 8192 bytes off that pipe into userspace and returns the first line; the rest of the reply sits in the decoder where select cannot see it, and the descriptor is now empty.

tmux frames each command's control-mode output with %begin / %end guards written in one burst, so the whole reply arrives together:

cmd-queue.c#L825-L832

Observed timeline from a raw os.read on the same descriptor:

t=0.0002  read(68)   %begin ... \n%end ... \n%session-changed $0 s\n
t=0.0003  read(50)   %begin ... \n\xe2\x90\x9e\n%end ... \n
t=0.4619  read(137)  %output %0 ...

The separator is fully buffered by the first or second readline(). What normally rescues the test is that third read: unrelated %output from the pane's shell painting its prompt re-arms select, and the loop then serves its remaining lines out of the userspace buffer. The test passes for a reason unrelated to the data it asserts on.

Reproduction

Deterministic once the read loop starts after the prompt-paint burst has landed, which is what a loaded CI runner produces. Injecting a stall between ControlMode.__enter__ and the loop, with everything else verbatim:

delay=0.0   0/4 failed
delay=0.15  1/4 failed
delay=0.3   3/4 failed

Proof the data was in hand at the moment select reported not-ready:

select NOT ready at iter=1 (last read = '%begin 1785458429 355 1')
  readline #0 returned in 7us  -> '␞'
  readline #1 returned in 3us  -> '%end 1785458429 355 1'
  readline #2 BLOCKED (fd empty)

select burned its full one-second budget while the answer was seven microseconds away.

CI signature

tests/test_control_mode.py:91: in test_control_mode_stdout_preserves_non_ascii_output
    assert ready, "timed out waiting for control-mode output"
E   AssertionError: timed out waiting for control-mode output
E   assert []
        line       = '%begin 1785370231 290 0\n'
        ready      = []

line is the loop variable holding the last line successfully read, so in both failures the test read %begin and the next select timed out. That is the signature of "read one line, then the descriptor went quiet", not of a slow tmux.

Ruled out, with evidence

• The locale.setlocale(LC_CTYPE, "C") the test performs. Removing it changes nothing: 4/4 failures at delay=0.3.
• tmux version. The mechanism is entirely Python-side. Two flakes landing on two of eight matrix cells is what randomness looks like.
• bufsize on Popen. Buffering mode is a write-side setting; measured no effect for 0, 1, or default.

What a fix needs

The library code is correct and should not change. text=True, encoding="utf-8" is deliberate — it is the regression this test exists to guard.

1. Drop select; read on a thread with a deadline. A reader thread pushes lines onto a queue.Queue, the test polls the queue with a timeout. Nothing straddles two buffers. Verified that a threaded reader still catches the original regression: with encoding="utf-8" removed it raises UnicodeDecodeError, so the coverage is preserved.
2. Or select on the raw descriptor via ctl.stdout.buffer.raw and decode in the test. Correct, but the test then owns partial-line handling and decoding.

Option 1 is preferred: the only reason select is present is to bound the wait, and a thread with a deadline does that without introducing a second buffer.

Related

tmux wait-for is the general answer for authored commands and is already wrapped as Server.wait_for() (src/libtmux/server.py#L617, cmd-wait-for.c#L34-L38), but it does not apply here: this test waits on control-mode protocol output rather than on a shell command it authored.