C and Pure parser accept invalid UTF-8 strings, the Java parser doesn't.
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Reproduce the Ruby example first, then read java/src/json/ext/ByteListTranscoder.java and compare the C, Pure, and Java parser behavior for invalid UTF-8. Clarify which behavior should match MRI before changing the parser; done means the implementations have an agreed, consistent result for this input.
Written by the indexing model from the issue text.
Description
I know, I know...if it's bad content it's bad content. But this represents a difference from MRI.
Here's the case, again a reduced version of one I got from @rkh:
# encoding: utf-8
require 'json'
x = "{\"foo\":\"\xC3\"}"
h = JSON.parse(x)
p h['foo']
p h['foo'].encoding
So basically there's a bad byte in a UTF-8 string, and the MRI version walks right by it and allows it to come through to the resulting parsed json structure.
I have a totally broken patch for this:
diff --git a/java/src/json/ext/ByteListTranscoder.java b/java/src/json/ext/ByteListTranscoder.java
index ed9e54b..a7e42ba 100644
--- a/java/src/json/ext/ByteListTranscoder.java
+++ b/java/src/json/ext/ByteListTranscoder.java
@@ -78,9 +78,10 @@ abstract class ByteListTranscoder {
return head;
}
if (head <= 0xbf) { // 0b10xxxxxx
- throw invalidUtf8(); // tail byte with no head
+ return head; //throw invalidUtf8(); // tail byte with no head
}
if (head <= 0xdf) { // 0b110xxxxx
+ if (pos + 1 > srcEnd) return head;
ensureMin(1);
int cp = ((head & 0x1f) << 6)
| nextPart();
Again, I'm not sure this is actually something that needs to be fixed, but because the MRI version of json does not blow up on this content, there's something to be addressed.
- Dominant language
- Ruby
- Stars
- 785
- Forks
- 390
- Avg merge
- 7h 14m
- Merged PRs (30d)
- 15
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ruby/json
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Maintainers usually reply within 1 day
-
Output formatting differences between jRuby 10.0.0.1 and Ruby 3.4 when converting Float to JSON.Possibly taken @headius claimed this 450 days ago. Openjruby
ruby/json#817 · 3 comments · 1 reaction · 1 assignee ·
Maintainers usually reply within 1 day
-
Don't cache JRuby runtime in static fieldsMay be free again @headius claimed this 727 days ago, and no pull request is open. Openjruby
ruby/json#707 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
new feature
Difficulty 5/5 Over a week Newbie friendliness 25/100
Maintainers usually reply within 1 day
Similar issues
-
🐛 bug 👩💻 équipe CE 🧑🏭 entreprises
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
betagouv/conseillers-entreprises#4808 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
yosiat/panko_serializer#250 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
cline/mcp-marketplace#3886 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100