Editor parity: TypeScript-source debugging and standalone LSP navigation

Inizia da docs/plans/editor-debugging-parity.md e dall’entry point sharpts-lsp, quindi esamina l’audit datato e le aree di validazione a cui fa riferimento. L’ambito del codice è dichiarato completo; ora, essere terminato significa eseguire la checklist manuale documentata per breakpoint ed esecuzione passo-passo in Visual Studio, Rider o in un’installazione funzionante di un debugger gestito.

Epic. Deliver first-class debugging of SharpTS-compiled programs from their original TypeScript source, then close the highest-value standalone-editor language-service gaps without duplicating tsserver in VS Code.

Design input: docs/plans/editor-debugging-parity.md. This epic incorporates the repository-validation corrections identified during review: PEPacker symbol preservation is a prerequisite; source provenance must survive AST transforms; navigation needs semantic binding identities in addition to spans; and LSP feature mode must be fixed at initialization unless dynamic capability registration is implemented.

Status update — 2026-07-28

Final implementation audit through #1318:

• M0–M4 are code-complete. The final PE/PDB pipeline, source provenance, supported semantic navigation domains, completeness-gated rename, workspace lifecycle, debugger state-machine metadata, structured interop code actions, and standalone-editor documentation have landed or are in #1318.
• Debugger polish is complete in metadata. Explicit async/generator/async-generator cases, captured/display/state-machine fields, stable generated naming, compiler-generated/non-user scaffolding attributes, standard state-machine attributes, Portable-PDB state-machine mappings, async suspension/resume records, managed-source language tagging, and debugger-resolvable CodeView paths are covered by automated tests.
• Standalone navigation is complete for the agreed domains. Type parameters, labels, namespace imports, qualified namespace members, type/value facets, cross-config/project-reference discovery, references, and safe rename use checker-backed identities. General object/class property-member navigation and rename remain deliberately deferred rather than returning partial results.
• Track D is complete. Immutable versioned snapshots, incremental changes and stale-update rejection, debounced cancellation, version-keyed analysis caches, forward/reverse dependency invalidation and importer republish, live diagnostics policy, safe metadata-loader reload, and interop quick fixes are implemented.
• Validation is green. The rebased branch passes 16,272 unit tests, the Test262 baseline (186 passed, 4 diagnostic skips), the TypeScript conformance baseline (45 passed), the VS Code extension compile, the language-server tool pack, and real initialize handshakes advertising quick fixes in both feature modes.

Remaining before closing the epic: run the documented manual breakpoint/stepping checklist in Visual Studio, Rider, or a working managed-debugger installation. Both current and prior stable netcoredbg builds failed on a Roslyn-generated C# control program in the test environment, so that local failure is not evidence about SharpTS's PDB. There is no remaining code priority within the agreed scope; exhaustive expression/type-node spans, general property/member navigation/rename, and full hoisted-local reconstruction remain explicit deferred breadth.

Outcomes

• sharpts --compile app.ts --debug emits a runnable assembly plus matching portable PDB whose documents and sequence points refer to the original .ts sources.
• Breakpoints bind in TypeScript, stepping follows executable TS statements, and user locals have useful names and lexical scopes.
• VS Code can compile and launch the current SharpTS file through the installed coreclr debug adapter.
• Standalone LSP clients can opt into document symbols, definition, references, and safe rename.
• VS Code's default remains interop-focused so SharpTS does not duplicate ordinary TypeScript navigation supplied by tsserver.
• Formatting and interpreter debugging remain deliberately out of scope.

Original baseline and constraints

This section records the pre-implementation baseline. The dated status update above is authoritative for current progress.

• The LSP currently registers sync, hover, completion, and signature help only.
• AST records have no source spans; Token.Start supplies UTF-16 source offsets.
• TypeMap is reference-keyed, so spans must live in a reference-keyed side table rather than participate in record equality.
• Parsing/compilation transforms replace or synthesize nodes (VarHoister, GeneratorArrowLifter, NestedFunctionLifter, destructuring lowerings, generator rewrites). Span provenance must explicitly cross those boundaries.
• ILEmitter overrides the shared statement dispatcher, so StatementEmitterBase.EmitStatement is not currently a universal sequence-point hook.
• TypeEnvironment maps names to types, not uses to declaration identities. Correct definition/references/rename requires a binding index, not only a position index.
• AssemblyReferenceRewriter comes from NickNa.PEPacker 1.0.2 and rebuilds the final PE without a debug directory. Symbol-aware rewriting must land before the debugger MVP can ship.
• The standalone server is sharpts-lsp; --standalone is already a compile/deployment option and must not be reused for LSP feature selection.

Track 0 — prove the final PE/PDB pipeline

This is the go/no-go gate and happens before broad parser instrumentation.

• Build a minimal PersistedAssemblyBuilder spike using GenerateMetadata(..., out pdbMetadata), PortablePdbBuilder, CodeView, and PDB checksum entries.
• Extend PEPacker (or replace the post-pass) so the final rewritten PE retains the debug directory and the PDB is built against the final metadata row counts/method-handle mapping.
• Release/consume the required PEPacker version if the fix remains external to this repository. (Not required: SharpTS uses a repository-local post-rewrite debug-directory injector and verifies method-row preservation.)
• Add a CI-safe test that opens the final assembly and PDB with System.Reflection.Metadata and verifies matching CodeView ID, documents, checksums, and sequence points.
• Preserve the existing SaveToBytes() contract for in-memory/test callers; introduce an explicit compilation-artifact result or symbol-aware Save path for PE + PDB output.

Gate: do not declare the debugger MVP unblocked until a PE that has passed through the real reference rewriter loads with its matching PDB.

Track A — source documents, spans, and transform provenance

Introduce a per-source model carried through parsing and modules, for example:

SourceDocument
  identity/path/URI
  original text or embedded-source payload
  checksum
  PositionMap
  SpanTable (reference equality)

• Add SourceSpan as half-open UTF-16 offsets and a SpanTable keyed by object reference.
• Return source-document/span data with parse results and retain it on ParsedModule, including virtual stdlib modules.
• Add parser helpers for production spans; preserve offsets when the parser converts tokens or splits >>/>>> tokens.
• Instrument declarations and executable statements first; expressions and type nodes follow for editor navigation. (Declarations/statements landed; broad expression/type-node span coverage remains incremental.)
• Define transform APIs such as CopySpan(original, replacement) and MarkHidden(synthetic).
• Apply provenance handling to VarHoister, GeneratorArrowLifter, NestedFunctionLifter, destructuring lowering, and other compiler/parser rewrites.
• Test exact spans, containment on source-backed nodes, and hidden/provenance behavior on transformed nodes. Synthetic nodes are not required to satisfy ordinary source-containment assertions.

Track B — TypeScript-source portable PDB debugging

B1. Breakpoints and statement stepping

• Add compile-only --debug / -g options and emit <output>.pdb beside the assembly.
• Emit normalized source paths and SHA-256 checksums for local files; embed source for virtual/embedded stdlib documents.
• Refactor statement dispatch to provide one real wrapper/hook across ordinary IL and all state-machine emitters.
• Define an executable-statement sequence-point policy. Skip type-only nodes; use hidden points for compiler-generated control flow; avoid duplicate points at the same IL offset.
• Add [Debuggable(Default | DisableOptimizations)] in debug builds only.
• Verify single-file, multi-module, async, generator, try/catch, loops, destructuring, and transformed var programs through PDB metadata tests. (Covered by explicit metadata tests for each listed domain.)
• Maintain a small manual debugger smoke checklist until an automated netcoredbg scenario is justified.

B2. Variables and lexical scopes

• Use LocalsManager as the primary seam for SetLocalSymInfo and ILGenerator.BeginScope/EndScope. (Implemented through LocalsManager.SymbolSink and the portable-PDB metadata writer.)
• Name only user-visible locals; keep spill/temp locals compiler-generated and hidden.
• Audit parameters, shadowed bindings, loop bindings, captured variables, display-class fields, and state-machine fields. (Covered across local-scope, display-class, captured-variable, and state-machine metadata tests.)
• Use stable compiler-generated display-class/type naming where it improves debugger presentation.
• Record the accepted v1 behavior for hoisted async/generator locals; full async-local reconstruction may require portable-PDB custom debug information.

B3. Just My Code and stepping polish

• Mark runtime helpers, reflection thunks, and non-user scaffolding with appropriate debugger/compiler-generated attributes. (Generated state-machine/display-class types and methods are consistently marked; runtime helpers and embedded stdlib remain outside user-code documents.)
• Keep user module methods user code while allowing embedded stdlib module methods to be skipped.
• Add proper state-machine/async stepping custom debug information as a follow-up after coarse breakpoint/step behavior is proven.
• Move optional DebuggerDisplay work to polish; it is not a blocker for source breakpoints.

B4. VS Code debugger UX

• Contribute a coreclr launch configuration/snippet and document the C# debug-adapter prerequisite.
• Add “SharpTS: Debug Current File”: save the dirty document, compile the exact saved source with --debug, retain runtimeconfig/dependencies, then call vscode.debug.startDebugging.
• Ensure the output/source path strategy works for imported modules and does not leave unbounded temporary output.
• Document Rider/Visual Studio and netcoredbg recipes.

Track C — standalone LSP navigation

C1. Feature-mode contract

• Add a dedicated sharpts-lsp --language-features interop-only|full option or equivalent initialization option.
• Default the standalone tool to full; have the VS Code extension explicitly launch it in interop-only mode.
• Decide whether mode changes require restart or implement LSP dynamic registration/unregistration. Do not claim workspace/configuration can change advertised capabilities by itself.
• Keep interop diagnostics, interop hover/completion/signature help, and SharpTS-specific code actions available in both modes.

C2. Position and semantic binding indexes

• Build position → narrowest source-backed node/token lookup over the span data.
• Introduce stable symbol identities and use→declaration bindings covering lexical scopes, hoisting, parameters, labels, type/value namespaces, declaration merging, imports, aliases, and re-exports. (Completed for the supported declaration/binding domains; general object/class property-member navigation remains deferred.)
• Prefer integrating binding capture with checker/environment resolution where practical rather than maintaining a divergent second implementation of TypeScript scoping.
• Preserve module/document provenance on every binding.

C3. Capabilities, in delivery order

• textDocument/documentSymbol — declarations/outlines; statement spans only.
• textDocument/definition — start with local bindings, then imports/re-exports and module-aware definitions. (For the indexed semantic domains; property/member navigation remains deliberately deferred.)
• textDocument/references — inverse index over the indexed module graph. (Includes configured projects, closed reverse importers, workspace folders, and project references.)
• textDocument/rename — produce WorkspaceEdit only when the complete affected symbol domain is known; refuse partial cross-module renames.
• SharpTS-specific code actions for interop diagnostics; carry structured diagnostic data so fixes do not depend on reparsing message text.

Property/member rename, semantic tokens, folding, inlay hints, and formatting are deferred unless demand justifies their separate complexity.

Track D — server performance and lifecycle

• Store document text and version and build requests from consistent snapshots.
• Debounce document checks and own a per-document/workspace cancellation source; cancellation must be observed between pipeline stages and inside long-running checker work where necessary.
• Cache text-version → tokens/AST/source document/span table/type-check result.
• Maintain forward and reverse module edges. A changed dependency must invalidate affected importers/open roots, not only its forward dependencies.
• Clear or republish diagnostics for every module affected by a graph update.
• Wire sharpts.diagnostics and feature initialization consistently across the standalone server and VS Code client.
• Reload AssemblyReferenceLoader safely when project/reference outputs change.
• Continue whole-file checking until measured workspaces prove incremental tree checking necessary.

Milestones

1. M0 — symbol pipeline proven: final rewritten PE + matching portable PDB metadata test.
2. M1 — breakpoints work: statement source model + B1; breakpoint and line stepping in local and imported .ts files.
3. M2 — usable debugging: local scopes/names, Just My Code, VS Code debug command, documentation.
4. M3 — standalone navigation: feature-mode initialization, document symbols, semantic binding index, definition/references/safe rename.
5. M4 — lifecycle and polish: versioned caches, reverse invalidation, cancellation, code actions, async stepping improvements.

Acceptance criteria

• A PDB metadata test verifies at least two TS documents, checksums, sequence points, named locals, lexical scopes, and matching final-PE CodeView identity.
• Breakpoints bind after the real PEPacker rewrite and on a program containing imports.
• Debug and non-debug builds remain runnable; non-debug output does not pay PDB costs.
• Existing in-memory compiler/test APIs remain usable.
• interop-only advertises or serves no general navigation results; full passes LSP integration tests for all four navigation capabilities.
• Rename never silently edits an incomplete cross-module reference set.
• Repeated edits cancel/debounce stale work, and a dependency signature change refreshes affected importer diagnostics.
• dotnet test, Test262, and TypeScript conformance baselines remain green/no-regression as appropriate.

Explicit non-goals

• Interpreter debugging or a bespoke SharpTS DAP adapter.
• A TypeScript formatter.
• Replacing tsserver in the default VS Code experience.
• Full async stepping fidelity in the first debugger milestone.
• True incremental type checking before profiling demonstrates a need.