Support reserved names in ProtoJSON deserialisers
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Domain
- backend-api-design
Research direction
Start by reviewing the ProtoJSON reserved-name requirement in this issue and the prior discussion in #3440. Identify how each affected ProtoJSON implementation represents reserved field names and handles unknown fields. Done means reserved names are discarded while truly unknown fields still fail by default, with coverage across the relevant implementations.
Written by the indexing model from the issue text.
Description
What language does this apply to?
My particular use case is in Go but it applies to all languages that implement ProtoJSON.
Describe the problem you are trying to solve.
Currently if a field is removed from a message, this is a wire-safe change for the binary serialiser (the field will be discarded by the deserialiser) but will trigger a parsing error for ProtoJSON.
That is, unless the "discard/ignore unknown" deserialiser option is enabled. However this will cause ALL unknown fields to be silently discarded. Which is a shame because the default behaviour is still desirable for truly unknown fields.
It's already possible to reserve field names with the reserve keyword, and the docs state that TextProto implementations may recognise this and discard unknown fields if it's a reserved name. The same docs also state "Runtime JSON parsing is not affected by reserved names.".
Describe the solution you'd like
It would be ideal if ProtoJSON implementations respected reserved names, by discarding unknown fields if they match a reserved name. This way it's possible to remove a field and have it safely deserialised by both binary and json implementations, as long as the field name is reserved. Truly unknown fields (not defined or reserved) still result in a parse error by default, catching actual errors ASAP.
NOTE: I am not requesting that unknown fields are preserved by ProtoJSON impls. The lack of preservation of reserved fields is completely acceptable (and even desirable imo).
Describe alternatives you've considered
The alternative (more of a workaround) is to enable the "discard unknown fields" ProtoJSON deserialiser option everywhere that could be affected by a removed field in the future (which is pretty much everywhere). As mentioned above this is bit of a sledgehammer fix and significantly weakens the safety checks of the deserialiser, which I'd like to avoid if at all possible.
Another alternative is to leave the field in the message definition and change the field doc comment to something like "OBSOLETE DO NOT USE", however this is confusing and easy to miss.
Additional context
Example:
Version 1
message MyMessage {
string name = 1;
string city = 2;
}
Version 2
message MyMessage {
reserved 2;
reserved city;
string name = 1;
}
This is a wire-safe change for the binary deserialiser, but it will cause the ProtoJSON deserialiser to fail with a unknown field "city" error. Ideally the "city": "my city" field would be discarded since the name is reserved (and thus "known").
Previous discussions around this (eg. #3440) mostly centred around the discrepancy between the binary & json deserialisation with default settings. I couldn't find an issue specifically around accounting for reserved names in ProtoJSON deserialisation.
- Dominant language
- C++
- Stars
- 72.2k
- Forks
- 16.4k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 342
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from protocolbuffers/protobuf
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
protocolbuffers/protobuf#31768 ·
Maintainers usually reply within 1 day
-
feature request packaging & distribution python
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
protocolbuffers/protobuf#31654 ·
Maintainers usually reply within 1 day
-
google-protobuf 0.36.2-release fails to compile for ARMv7: @progbits rejected in global_asm!Possibly taken @farisbasha claimed this 1 day ago. Openbug help wanted rust
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
protocolbuffers/protobuf#31619 · 3 comments ·
Maintainers usually reply within 1 day
-
Null pointer passed to memcpy in lupb_decode() when the second argument is nilPossibly taken A pull request linked to this issue is open or already merged. Openbug help wanted lua
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
protocolbuffers/protobuf#31392 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
protocolbuffers/protobuf#31032 ·
Maintainers usually reply within 1 day
All issues in protocolbuffers/protobuf
Similar issues
-
bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
microsoft/DirectXShaderCompiler#9155 ·
Maintainers usually reply within 2 days
-
Errors in 1.20 proseOpenCampaign DiD Campaign EI Campaign HttT Campaign TDG Campaign TSG Campaign WoF Enhancement Help Prose Stable 1.20
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
wesnoth/wesnoth#11749 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
trabucayre/openFPGALoader#767 ·
Maintainers usually reply within 1 day
-
agent:Windows bug in progress pri:medium sev:low testing windows
Difficulty 1/5 Under an hour Newbie friendliness 91/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
godotengine/godot#125651 ·
Maintainers usually reply within 1 day