Numeric character references (A, A) are not decoded unless htmlEntities is enabled

Open
#892 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

  • #893 by @guocn — closed without merging

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
64/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, node.js
Domain
backend

Research direction

Start by reading OrderedObjParser and its EntityDecoder construction, especially the numericAllowed option mentioned in the issue; then inspect the existing entity-decoding tests. Reproduce the examples for element text and attribute values with default options, and run the relevant parser tests. Done means numeric XML character references decode without enabling htmlEntities, while the existing named-entity behavior remains covered.

Written by the indexing model from the issue text.

Description

With default options, processEntities is true and the five predefined XML entities (<, >, &, ", ') are decoded as expected. Numeric character references, however, are silently left as literal text unless htmlEntities: true is passed:

const { XMLParser } = require('fast-xml-parser');

const parser = new XMLParser(); // processEntities defaults to true

parser.parse('<r>&lt;&gt;&amp;</r>');
// { r: '<>&' }  -- named entities decoded

parser.parse('<r>&#65;&#x42;</r>');
// { r: '&#65;&#x42;' }  -- numeric refs NOT decoded (expected: { r: 'AB' })

parser.parse('<r a="&#65;"/>', { ignoreAttributes: false });
// attributes affected the same way

new XMLParser({ htmlEntities: true }).parse('<r>&#65;&#x42;</r>');
// { r: 'AB' }  -- only decoded when htmlEntities is on

Numeric character references are not HTML entities - they are part of core XML. Per the XML 1.0 spec (section 4.1, "Character and Entity References"), a well-formed XML processor must recognize &#...; and &#x...; references in content and attribute values, independent of any HTML entity table.

Looking at the source, this seems to come from OrderedObjParser wiring numericAllowed to the htmlEntities option when constructing the EntityDecoder:

this.entityDecoder = new EntityDecoder({
  namedEntities: { ...namedEntities, ...externalEntities },
  numericAllowed: this.options.htmlEntities, // <-- ties XML char refs to the HTML entities option
  ...
});

So a user who parses XML with default options (and htmlEntities defaults to false) gets raw &#65; sequences in their data, while named XML entities are decoded. I ran into this with a feed that encodes non-ASCII characters as numeric refs.

Tested on fast-xml-parser 5.11.2 (npm latest) with Node 22. The same behavior exists in 4.x, so it may be intentional - but if so, it would be great to have it documented next to processEntities, since the current default contradicts XML spec compliance.

Dominant language
JavaScript
Stars
3.1k
Forks
410
Avg merge
4d 15h
Merged PRs (30d)
8

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NaturalIntelligence/fast-xml-parser

All issues in NaturalIntelligence/fast-xml-parser

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.