Encode mishandles unrepresentable Shift_JIS characters and FB_PERLQQ
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start with t/pp_07_coder_check_value.t and the loaded t/tests/07_coder_check_value_euc-jp.t regression, then trace the PerlOnJava Encode implementation used by both JVM and interpreter backends. Reproduce the behavior with system Perl, add a focused regression for Shift_JIS replacement and Encode::FB_PERLQQ, and verify both backend results plus the upstream suite.
由索引模型根据 Issue 内容生成。
描述
Summary
Text-CSV-Encoded 0.25's unchanged EUC-JP encoding test passes under system Perl but fails on both PerlOnJava backends. The failure shows that encoding an unrepresentable character to Shift_JIS does not honor Encode's replacement/check behavior: output loses CSV quoting, and Encode::FB_PERLQQ yields U+FFFD instead of the expected \\x{2460} representation.
Reproduction
- CPAN random tester run:
20261002-182937-4648 - Distribution:
Text-CSV-Encoded0.25 - Test:
t/pp_07_coder_check_value.t, which loadst/tests/07_coder_check_value_euc-jp.t - Environment: macOS arm64, Java 24.0.2, PerlOnJava 5.44.1 (build commit
78521d8d0, 2026-10-02)
The test parses EUC-JP input and combines fields as Shift_JIS. It checks default replacement behavior and then sets encode_check_value(Encode::FB_PERLQQ) before repeating the operation.
System Perl 5.42.2 passes the complete upstream suite (14 files, 255 tests).
PerlOnJava fails both assertions on both backends:
not ok 1
got: '...,?'
expected: '...,"?"'
not ok 2
got: '...,\\x{FFFD}'
expected: '...,"\\x{2460}"'
The result was reproduced with the archived distribution and current PerlOnJava build on both JVM and interpreter backends. The distribution has no XS/C source; its xs_* tests exercise the separate Text::CSV dependency.
Expected behavior
Match system Perl for Shift_JIS encoding of characters that cannot be represented, including the default replacement and FB_PERLQQ check behavior. Preserve the resulting CSV field quoting.
Add a focused project regression for the encoding/check behavior, validate it with system Perl, then verify it on both PerlOnJava backends and rerun the upstream test.
Related issue
Related to #1445, which tracks PerlOnJava Encode compatibility, though that issue covers canonical charset names rather than unrepresentable-character handling.
- 主要语言
- Perl
- 星标
- 64
- 派生
- 7
- 平均合并
- 4 小时 27 分钟
- 30 天内合并 PR
- 178
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
fglock/PerlOnJava 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
fglock/PerlOnJava#1651 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 72/100
fglock/PerlOnJava#1579 ·
维护者通常 1 天内回复
-
area:cpan-port area:unicode bug
难度 2/5 1-3 小时 新手友好度 88/100
fglock/PerlOnJava#1341 ·
维护者通常 1 天内回复
-
area:backend area:cpan-port area:parser bug
难度 5/5 一周以上 新手友好度 25/100
fglock/PerlOnJava#1659 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 25/100
fglock/PerlOnJava#1657 · 2 条评论 ·
维护者通常 1 天内回复
查看 fglock/PerlOnJava 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 85/100
gugod/App-perlbrew#879 ·
-
难度 2/5 1-3 小时 新手友好度 76/100
RotherOSS/otobo#6205 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
yuki-kimoto/SPVM-IO#30 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
RotherOSS/servicecatalog#28 ·
-
bug
难度 2/5 1-3 小时 新手友好度 70/100
convos-chat/convos#977 ·