[Tests] serve/test_fatal_recovery.py: the fake engine's .cmd is written in UTF-8 while cmd.exe reads the console code page - 8 errors in UTF-8 mode

Open Beginner friendly
  • Is something wrong?
#5,632 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
1-3 hours
Newbie friendliness
85/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing

Research direction

Read serve/test_fatal_recovery.py, especially FatalRecovery.setUp where the Windows batch file is written, and check how that file is launched by the test. Run python -X utf8 -m unittest serve.test_fatal_recovery on Windows with a non-ASCII repository path and Korean console code page to reproduce the errors. Done means all eight tests pass in UTF-8 mode.

Written by the indexing model from the issue text.

Description

needs-format

Strata app 0.2.12 (146df3c4) · Python 3.14 · OS Windows 11, Korean locale (console code page cp949)

Reproduce — four runs, one machine, one file

where the repo lives how the test was started result
C:\Users\lowho\Documents\기본 프로젝트\Strata python -m unittest serve.test_fatal_recovery Ran 8 tests in 5.720s → OK
same PYTHONUTF8=1 python -m unittest serve.test_fatal_recovery Ran 8 tests in 4.275s → FAILED (errors=8)
same python -X utf8 -m unittest serve.test_fatal_recovery FAILED (errors=8)
C:\Users\lowho\AppData\Local\Temp\opencode\wt-head (ASCII path) python -X utf8 -m unittest serve.test_fatal_recovery Ran 8 tests in 5.771s → OK

All eight errors are the same one, raised in setUp before a test body runs, so the file reports nothing about its real subject:

serve.server.EngineDied: the engine exited before it was ready
the engine process ended: exit code 1

Why

FatalRecovery.setUp (L38-40) runs the fake engine through a batch file, because StrataEngine starts [exe, "--serve", *args] (serve/server.py L1210) and a .py cannot take that:

if os.name=='nt':   # no shebangs on Windows: a .cmd runs the same fake engine
    py.write_text(FAKE);self.script=self.root/'native.cmd'
    self.script.write_text(f'@"{sys.executable}" "{py}" %*\r\n')

write_text() uses the interpreter's default encoding: cp949 normally, UTF-8 in UTF-8 mode (PYTHONUTF8=1, -X utf8). cmd.exe reads batch files in the console's code page. So in UTF-8 mode the batch file names sys.executable in bytes cmd does not read, and only when that path is non-ASCII — which is why an ASCII path hides it.

One file, two encodings, same content:

body = '@"' + sys.executable + '" -c "print(1)"\r\n'      # sys.executable on a Korean path
for name, enc in (('utf8.cmd', 'utf-8'), ('cp949.cmd', 'cp949')):
    (d / name).write_bytes(body.encode(enc))
    print(name, subprocess.run(['cmd', '/c', str(d / name)], capture_output=True).returncode)

utf8.cmd  file-enc=utf-8  rc=1  out=b''  err=b'\xc1\xf6\xc1\xa4\xb5\xc8 \xb0\xe6\xb7\xce...'   ("지정된 파일을 찾을 수 없습니다")
cp949.cmd file-enc=cp949  rc=0  out=b'1\r\n'

UTF-8 mode is exactly where the non-UTF-8 Windows fixes elsewhere in this project push people, so the combination is not exotic.

Fix

Write the batch file the way cmd.exe reads it. Probed with the real StrataEngine and the real fake engine, on the Korean path, in UTF-8 mode:

test_A_default_encoding (what the test does today) ... ERROR   -> EngineDied: the engine exited before it was ready (exit code 1)
test_C_console_code_page (encoding="mbcs")          ... ok

         if os.name=='nt':   # no shebangs on Windows: a .cmd runs the same fake engine
             py.write_text(FAKE);self.script=self.root/'native.cmd'
-            self.script.write_text(f'@"{sys.executable}" "{py}" %*\r\n')
+            # cmd.exe reads a batch file in the console's code page, not in UTF-8: a
+            # UTF-8 mode interpreter would otherwise write a path cmd cannot find
+            self.script.write_text(f'@"{sys.executable}" "{py}" %*\r\n', encoding='mbcs')

py.write_text(FAKE) can take encoding="utf-8" as well; the fake engine's own source is ASCII today, so it is not what breaks.

Dominant language
C++
Stars
11.6k
Forks
1k
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Niko1221/Strata

All issues in Niko1221/Strata

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.