[Bug]: Non-ASCII input inside an unfinished CSI loses visible recovery text from history

Abierto
  • ¿Algo no está bien?
#36,015 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
typescript, wasm
Área
backend

Línea de trabajo

Start in apps/server/src/terminal/Manager.ts, at sanitizeTerminalHistoryChunk and isCsiFinalByte, which decide what history keeps after an unfinished CSI. The attached repro-csi-unicode.mjs script shows the failing case: the input A, ESC, [, U+754C, n, B keeps only AB in history while the live engine renders A�nB. Done means replaying that input matches the live output, and the valid A CSI 31m B control still keeps its bytes.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

vouch:denounced

Summary

The history scanner keeps treating non-ASCII input as part of an unfinished control sequence after the bundled terminal has already recovered and resumed printing text. A later status-report final byte makes the scanner discard that visible text. A deterministic scanner and engine probe reproduced output present live but absent on replay.

Before submitting

  • Related issues: searched open and closed issues and PRs; related history is listed below, with no exact current duplicate found.
  • Removed credentials, tokens, private paths, and sensitive data.

Area

apps/server

Observed behavior

For A ESC [ 界 n B (UTF-8 hex 411b5be7958c6e42), the bundled Ghostty engine renders A�nB and generates no reply. The history scanner instead keeps AB, so replay loses both the recovery character and the ordinary n that the live engine displayed. The valid A CSI 31m B control retains its bytes and renders AB both live and on replay.

Reproduction

  1. Use a checkout of the baseline below. Save the script as repro-csi-unicode.mjs outside the checkout and run node repro-csi-unicode.mjs /path/to/checkout.
  2. The script runs an unfinished CSI followed by U+754C and printable sentinels, plus a valid color-sequence control through the unchanged production definitions extracted into an in-memory module, then compares live bytes and replayed history in the bundled Ghostty WASM. It needs Node with node:module.stripTypeScriptTypes, Git, and the checked-in WASM artifacts; installed application dependencies are unnecessary.

The result is deterministic. Every two-chunk scanner split and every raw-byte split of the live engine input produced the same result, including all three bytes of U+754C. The script creates a disposable directory, points HOME, TMPDIR, and T3CODE_HOME into it, removes it on exit, and leaves the checkout untouched. Exit 1 means the defect was reproduced, exit 0 means expected behavior, and exit 2 means setup or a control failed. The recorded run exited 1 after all controls passed. This is a unit-style scanner/engine reproduction, not a running application or native-device test.

Reproduction script: repro-csi-unicode.mjs 
import fs from 'node:fs';
import os from 'node:os';
import {execFileSync} from 'node:child_process';
import path from 'node:path';
import { stripTypeScriptTypes } from 'node:module';
const scratch=fs.mkdtempSync(path.join(os.tmpdir(),'terminal-history-repro-'));
let status=2;
try {
 for(const name of ['HOME','TMPDIR','T3CODE_HOME']){const dir=path.join(scratch,name.toLowerCase());fs.mkdirSync(dir);process.env[name]=dir;}
 if(!process.argv[2])throw Error('Pass the checkout path as the only argument');
const root = path.resolve(process.argv[2]);
console.log('Git', execFileSync('git',['--version'],{encoding:'utf8'}).trim());
console.log('T3 commit',execFileSync('git',['-C',root,'rev-parse','HEAD'],{encoding:'utf8'}).trim());
console.log('Node',process.version);
const source=fs.readFileSync(path.join(root,'apps/server/src/terminal/Manager.ts'),'utf8');
const scanSource=source.slice(source.indexOf('function isCsiFinalByte('),source.indexOf('\nfunction ',source.indexOf('function sanitizeTerminalHistoryChunk(')+10));
const boundedSource=source.slice(source.indexOf('export class BoundedTerminalHistory'),source.indexOf('// Pop beyond Ghostty'));
const contract=fs.readFileSync(path.join(root,'packages/contracts/src/terminal.ts'),'utf8');
const tailSource=contract.slice(contract.indexOf('const historyEncoder'),contract.indexOf('const TrimmedNonEmptyStringSchema'));
const clientSource=fs.readFileSync(path.join(root,'packages/client-runtime/src/state/terminalOutput.ts'),'utf8').replace('import { retainedTerminalHistory } from "@t3tools/contracts";','');
const extracted=stripTypeScriptTypes('const DEFAULT_HISTORY_BYTE_LIMIT=8*1024*1024; const MAX_HISTORY_CHUNK_LENGTH=16*1024;\n'+boundedSource+tailSource+scanSource+'\n'+clientSource+'\nexport {sanitizeTerminalHistoryChunk};',{mode:'strip'});
const api=await import('data:text/javascript;base64,'+Buffer.from(extracted).toString('base64'));
const scan=(chunks)=>{let kept='',pending=''; for(const s of chunks){const r=api.sanitizeTerminalHistoryChunk(pending,s);kept+=r.visibleText;pending=r.pendingControlSequence;}return {kept,pending};};
const wasm=await WebAssembly.instantiate(fs.readFileSync(path.join(root,'apps/web/src/terminal/ghostty/vendor/ghostty-vt.wasm')),{env:{log:()=>{}}});
const e=wasm.instance.exports;
const call=(n,...a)=>e[n](...a);
const mem=e.memory;
const alloc=(n)=>call('ghostty_wasm_alloc_u8_array',n);
const free=(p,n)=>call('ghostty_wasm_free_u8_array',p,n);
const view=()=>new DataView(mem.buffer);
const bytes=(p,n)=>new Uint8Array(mem.buffer,p,n);
const decode=(p,n)=>new TextDecoder().decode(bytes(p,n));
const check=(r)=>{if(r!==0)throw Error('Ghostty result '+r);};
let replies='';
const trampoline=await WebAssembly.instantiate(fs.readFileSync(path.join(root,'apps/web/src/terminal/ghostty/vendor/ghostty-write-pty.wasm')),{env:{t3_write_pty:(_t,_u,p,n)=>{replies+=decode(p,n);}}});
const table=e.__indirect_function_table, callback=table.length;
table.grow(1,trampoline.instance.exports.ghostty_write_pty);
const revSlot=alloc(8);check(call('ghostty_build_info',10,revSlot));console.log('Ghostty',decode(view().getUint32(revSlot,true),view().getUint32(revSlot+4,true)));free(revSlot,8);
function terminal(chunks){
 const opts=alloc(8),slot=alloc(4);bytes(opts,8).fill(0);view().setUint16(opts,80,true);view().setUint16(opts+2,24,true);check(call('ghostty_terminal_new',0,slot,opts));const t=view().getUint32(slot,true);free(opts,8);free(slot,4);
 call('ghostty_terminal_set',t,0,1);call('ghostty_terminal_set',t,1,callback);replies='';
 for(const s of chunks){const b=typeof s==='string'?Buffer.from(s):s;if(!b.length)continue;const p=alloc(b.length);bytes(p,b.length).set(b);call('ghostty_terminal_vt_write',t,p,b.length);free(p,b.length);}
 const pos=alloc(8);check(call('ghostty_terminal_get',t,3,pos));const x=view().getUint16(pos,true);check(call('ghostty_terminal_get',t,4,pos));const y=view().getUint16(pos,true);check(call('ghostty_terminal_get',t,12,pos));const title=decode(view().getUint32(pos,true),view().getUint32(pos+4,true));free(pos,8);
 // WASM C ABI passes the formatter options struct by reference.
 const fopts=alloc(24),fslot=alloc(4),output=alloc(8192),written=alloc(4);bytes(fopts,24).fill(0);view().setUint32(fopts,24,true);bytes(fopts+8,2).fill(1);check(call('ghostty_formatter_terminal_new',0,fslot,t,fopts));const f=view().getUint32(fslot,true);check(call('ghostty_formatter_format_buf',f,output,8192,written));const text=decode(output,view().getUint32(written,true));call('ghostty_formatter_free',f);free(fopts,24);free(fslot,4);free(output,8192);free(written,4);
 const result={reply:replies,text,title,x,y};call('ghostty_terminal_free',t);return result;
}

const assert=(condition,message)=>{if(!condition)throw Error('Setup/control failed: '+message);};
const defects=[];
function allSplits(input){
 const expected=scan([input]),baseline=terminal([input]);
 for(let i=0;i<=input.length;i++)assert(JSON.stringify(scan([input.slice(0,i),input.slice(i)]))===JSON.stringify(expected),'scanner split '+i);
 const raw=Buffer.from(input);
 for(let i=0;i<=raw.length;i++)assert(JSON.stringify(terminal([raw.subarray(0,i),raw.subarray(i)]))===JSON.stringify(baseline),'Ghostty byte split '+i);
 return {kept:expected.kept,pending:expected.pending,live:baseline,replay:terminal([expected.kept])};
}

const control=allSplits('A\x1b[31mB');
assert(control.kept==='A\x1b[31mB'&&control.live.text==='AB'&&control.replay.text==='AB','valid CSI style');
const input='A\x1b[界nB',result=allSplits(input);
assert(result.live.text==='A\ufffdnB'&&result.live.reply===''&&!result.pending,'live parser recovery');
console.log(JSON.stringify({input,utf8Hex:Buffer.from(input).toString('hex'),expectedReplayText:result.live.text,...result}));
if(result.replay.text!==result.live.text)defects.push('visible recovery text lost');

 status=defects.length?1:0;
}catch(error){console.log('Setup error:',error.message.replaceAll(scratch,'<scratch>'));status=2;}
finally{fs.rmSync(scratch,{recursive:true,force:true});}
console.log(status===0?'RESULT: expected behavior':status===1?'RESULT: defect reproduced':'RESULT: inconclusive');
process.exit(status);

Expected behavior

Filtering history should preserve the visible output resulting from the bundled parser’s recovery, rather than swallowing printable text as a status query that the engine never dispatched. ECMA-48 section 5.4 gives separate byte ranges for CSI parameters, intermediates and final bytes; a Unicode code point outside those ranges cannot be blindly treated as another parameter. Error recovery itself is implementation-dependent, so the expected display here is established by the pinned Ghostty engine, not a claim that the standard prescribes its replacement character.

Impact and scope

Impact: P2 - A process producing a truncated CSI followed by non-ASCII text can leave saved scrollback missing text visible live. This is bounded to malformed or interrupted control streams. The report does not ask the terminal to preserve the original CJK character beyond what its live parser displays.

Form impact: Minor bug or occasional failure. Workaround: NOT_ENOUGH_INFO — no general end-user workaround was demonstrated for arbitrary process output.

Version

T3 Code commit 59e64eef881632a23f89c4138bcf37f23d92bf93.

Runtime and platform

Node v26.12.1; Git 2.56.0; macOS arm64 (Darwin 27.2.0). Vendored libghostty-vt WASM build info reports 9f62873bf195e4d8a762d768a1405a5f2f7b1697, matching native/libghostty-vt/VERSION. No browser, application server, real PTY, or native mobile device was started.

Interface

Library: server terminal-history scanner and bundled Ghostty C ABI.

Earliest failure stage

Input parsing or validation.

Relevant output or evidence

Actual behavior

The following display uses named markers for ESC, NUL, BEL, DEL, and U+009C; reproduction inputs in the script above retain executable escape notation.

Git git version 2.56.0
T3 commit 59e64eef881632a23f89c4138bcf37f23d92bf93
Node v26.12.1
Ghostty 9f62873bf195e4d8a762d768a1405a5f2f7b1697
{"input":"A<ESC>[界nB","utf8Hex":"411b5be7958c6e42","expectedReplayText":"A�nB","kept":"AB","pending":"","live":{"reply":"","text":"A�nB","title":"","x":4,"y":0},"replay":{"reply":"","text":"AB","title":"","x":2,"y":0}}
RESULT: defect reproduced
Evidence
  • Expected source: the documented restoration/retention contract and pinned parser behavior described above.
  • Failure source: the script output above, running the unchanged production definitions and checked-in WASM.
  • Evidence provenance: observed.
  • Local verification: reproduced.
  • Reproduction completeness: complete for the scanner/retention and engine boundary; no full-client behavior is claimed.

apps/server/src/terminal/Manager.ts#L1144-L1328 contains the scanner exercised by the probe. apps/server/src/terminal/Manager.ts#L2321-L2331 passes each PTY output chunk to it and appends its kept bytes to session history.

Restoration check

The script should exit 0: replay preserves the same visible recovery text as live parsing, and the valid color-sequence control continues to retain its bytes and display correctly.

Additional context

References

AI-assisted report; verified with the script and tools listed above. No source changes, fixes, or PRs are included.

Lenguaje dominante
TypeScript
Estrellas
24.8k
Forks
6.4k
Merge medio
59 min
PR fusionados (30 d)
418

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de pingdotgg/t3code

Todos los issues de pingdotgg/t3code

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.