Control characters in run text are deleted on save, so Segment.BeginOffset and run boundaries shift after reload
还没有人认领这个 Issue。
评估
调研方向
Start by reading StripInvalidXmlChars in TsStringSerializer and TsStringUtils.NormalizeNfd(ITsString), then compare the suggested behavior with #414. Run the named ControlCharacterOffsetDesyncTests and the TsStringSerializerTests round-trip to confirm the offset shift. Done means serialization and normalization preserve text length by replacing invalid XML characters with U+FFFD, or the chosen input policy is covered by tests.
由索引模型根据 Issue 内容生成。
描述
Characters XML cannot store (U+0000–U+0008, U+000B, U+000C, U+000E–U+001F, U+FFFE, U+FFFF) are accepted by the model, but TsStringSerializer silently deletes them on write (StripInvalidXmlChars). After reload, anything that stores a character index into the text points at the wrong character.
Repro (master 0bc98fb0, net462 and net8.0):
- Take a paragraph
One a<U+0001>b two. Three four.whose second segment hasBeginOffset13 ("Three"). Save, close and reopen the project. The text is now 23 characters instead of 24, andBeginOffset13 points at "hree ". The next parse or edit misplaces analyses. - The same happens with a plain string: runs
ab<U+0001>cd(en) +EFGH(es) serialize and deserialize with the es run starting at 4 instead of 5.
ktptOffset is not affected: it is a baseline shift in millipoints, not a character index.
Suggested fix: make writes length-preserving by replacing these characters with U+FFFD instead of deleting them, and apply the same replacement at input in TsStringUtils.NormalizeNfd(ITsString). This is the approach #414 takes for lone surrogates. Whether to replace them or reject them at input is a maintainer decision. Failing tests are ready (ControlCharacterOffsetDesyncTests, and a TsStringSerializerTests round-trip) and can come with the fix PR.
Related: #409, #412, #414.
- 主要语言
- C#
- 星标
- 3
- 派生
- 15
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
sillsdev/liblcm 的其他 Issue
-
Saving run text that contains a lone (unpaired) surrogate throws in TsString normalization可能已有人在做 @johnml1135 于 2 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 30/100
-
Text props: an unknown writing-system handle is saved as ws="", which no reader can load可能已有人在做 @johnml1135 于 2 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 25/100
-
Text property XML: writer emits properties and values its own reader rejects (record fails to load)可能已有人在做 @johnml1135 于 2 天前认领。 未关闭
难度 5/5 一周以上 新手友好度 25/100
-
难度 4/5 3-5 天 新手友好度 68/100
-
难度 4/5 3-5 天 新手友好度 38/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 74/100
juandresrodca/AdminForge#54 ·
-
要トリアージ 辞書
难度 1/5 1-3 小时 新手友好度 86/100
yksr-melt/Meltype#1625 · 1 个 reaction ·
维护者通常 1 天内回复
-
Docs typo未关闭
难度 1/5 1 小时以内 新手友好度 78/100
MonoGame/docs.monogame.github.io#328 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
space-wizards/RobustToolbox#7197 ·
维护者通常 1 天内回复
-
documentation high priority
难度 2/5 1-3 小时 新手友好度 76/100
lachydotmcg/capri-desktop#201 ·
维护者通常 1 天内回复