JRuby: UtfHelpper.writeCharToUtf8 cannot handle unicode supplementary character · sparklemotion/nokogiri#2410

(9 comments) (0 reactions) (0 assignees)Ruby (5,615 stars) (806 forks)batch import

blockedhelp wantedplatform/jrubytopic/encoding

説明

https://github.com/sparklemotion/nokogiri/blob/55029bfba481338825c99e78af2b182b1cc49e04/ext/java/nokogiri/internals/c14n/UtfHelpper.java#L51

since the Canonicalizer process input String character by character. Java uses 16 bits to represent a character; when the input string contains Unicode characters whose code pen are larger than 0Xffff（65535) it will be split into two char, since neither char will not be
recognized, the Unicode characters will be transferred to 2 ??(3f) instead.

for example, if I want to canonicalize an input that contains 𡏅 via c14n, in the output, 𡏅 will be replaced with ??

コントリビューターガイド

技術スタック: java
領域: backend
Issue 種別: bug
難度: 3
推定時間: 1-3 hours
活動状況: active
明確さ: clear
前提条件: Understanding of Unicode and Java char encoding
初心者向け度: 40
調査方針: The issue is in UtfHelpper.java, specifically the method writeCharToUtf8 that processes characters individually, failing for supplementary characters (code points > U+FFFF). The fix should handle surrogate pairs, iterating over the string using codePointAt or similar. Refer to the file line 51 in the linked source.