Microsoft/vscode

Text replace with surrogate pairs not reliable

Open

#248268 opened on May 7, 2025

View on GitHub
 (10 comments) (0 reactions) (1 assignee)TypeScript (74,848 stars) (10,221 forks)batch import
bugconfirmedhelp wantedsearch-replace

Description

Type: Bug

  1. Open a new file
  2. Paste "🌟 🌈 🚀 🎉"
  3. Find/Replace (ctrl+shift+F menu) Find "🌟 🌈 🚀 🎉" Replace "🎉 🚀 🌟 🌈"

This results in "� 🚀 � 🌈"

This issue was found while investigating the same behavior happening after applying an edit like so from extension code:

// doc is a TextDocument whose current contents include surrogate pairs
// as in the above example
edit = vscode.WorkspaceEdit()
edit.replace(doc.uri, vscode.Range(0, 0, doc.lineCount, 0), contentsWithSurrogatePairs)
await vscode.applyEdit(edit)
await doc.save()

In contrast, this edit reliably yields the desired result, but cannot perform targeted replacements to a sub-range of the document.

edit.createFile(doc.uri, { overwrite: true, contents: Buffer.from(contentsWithSurrogatePairs) });

VS Code version: Code - Insiders 1.100.0-insider (abe4aab6a7ddb38f4a83c38b53cb4f28dddf0c97, 2025-05-06T20:36:33.967Z) OS version: Windows_NT x64 10.0.26100 Modes:

Extensions: none

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt551:31249597
vscod805:30301674
c4g48928:30535728
962ge761:30841072
dsvsc014:30777825
dsvsc015:30821418
h48ei257:31000450
pythontbext0:30879054
cppperfnew:30980852
dwnewjupyter:31046869
pythonrstrctxt:31093868
nativeloc1:31118317
e80f6927:31120813
dwcopilot:31158714
5b1c1929:31184661
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
9064b325:31222308
copilot_t_ci:31222730
g012b348:31231168
pythoneinst12:31251391
bgtreat:31252123
c7cif404:31294349
31787653:31262186
3e8i5726:31271747
pythonpulldiag:31287486
6gi0g917:31259950
61ibe868:31282443
996jf627:31264550
pythonrdcb7:31268811
usemplatestapi:31297334
i53e4855:31297587
747dc170:31275146
pythoncompactctx:31296835
g20af354:31278749
9hi5e907:31296557
pylancecolor:31288664
generatesymbolt:31280541
convertfstringf:31280702
gendocf:31280703
3g1ci329:31285109
pylancequickfixt:31296868

Contributor guide

Tech stack
typescriptelectronnodejs
Domain
desktop
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
blocked
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
basic knowledge of Unicode surrogate pairsfamiliarity with VS Code extension API
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Investigate the text replace mechanism in VS Code, focusing on the `edit.replace` API. Look into the editor's text model implementation (likely in `src/vs/editor/common/model/`). The issue indicates that direct file content replacement works, suggesting the problem lies in the range replace logic. Check if surrogate pair boundaries are respected. Also review similar issues or comments for clues. Consider testing with browser based Unicode handling.