One HTML parse contract: encoding, wrapper, flags and error policy in one place

Open
#5,597 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
php, wordpress
Domain
backend

Research direction

Start with the top-level Support parsing entry points and HtmlSerialization::loadHtml() / documentHtml(); the issue identifies 84 load sites across 41 files and 4 loadXML() files. Read the current parsing and serializer paths, then trace the first slice’s candidate call sites and run the Lab gate plus the A/B output_sha256 comparison. Done means each slice uses the shared contract and reports any pages whose output changes.

Written by the indexing model from the issue text.

Description

Problem

The engine has no single place that defines how HTML is parsed. Each call site defines its own parse:

Measure Count
Load sites 84, across 41 files
Encoding-hint spellings 2: '<?xml encoding="UTF-8">' (29 sites) and '<?xml encoding="utf-8" ?>' (30). Another 20 sites get the first spelling through HtmlSerialization::utf8Document()
Wrappers chosen per site 3: <body> (34), <html><body> (11), <div> (11)
Flags Chosen per site: NOIMPLIED at 41 sites, NONET at 33
Error policies 3: HtmlSerialization::loadHtml() (63), @-suppression (10), LIBXML_NOERROR flags (11)
DOMDocument construction 2 forms: new DOMDocument() (64) and new DOMDocument('1.0', 'UTF-8') (8)

The encoding hint is added in one place and removed in another. Each site prepends the hint when it parses. Only HtmlSerialization::documentHtml() removes it, at serialization time. As a result, the tree a caller receives depends on that call site's choices.

A local probe on libxml 2.9.13 and PHP 8.5 shows this. With LIBXML_HTML_NOIMPLIED and no wrapper element, the hint is a top-level node. A caller that walks or serializes top-level nodes therefore emits <?xml encoding="…"> along with the content, and which spelling appears depends on the call site. With a wrapper, the trees under the wrapper are identical for both spellings and both constructor forms.

Direction

Add one parse contract in top-level Support, so every layer can reach it:

  • Two entry points. One parses a fragment and one parses a document. The flags follow from that choice, and LIBXML_NONET is always on.
  • It owns encoding. It adds the hint, parses, and removes the hint right away, so every caller receives a clean tree.
  • One error policy.
  • An optional wrapper. Callers keep the <body>, <div> or <html><body> context they need.

With that in place, documentHtml() no longer strips anything, and call sites only state what they're parsing. When the PHP floor reaches 8.4, Dom\HTMLDocument (a native HTML5 parser with native UTF-8) can replace libxml behind this one seam, and the encoding hint goes away entirely.

Slices

  1. Add the contract, and move the sites whose parse is provably unchanged: the same tree under a wrapper, or no top-level walk. Output must stay byte-identical.
  2. Move the sites where the hint currently leaks into the tree. Any output change is a fix, and each PR lists the pages whose output changed.
  3. Move loadXML() (4 files) behind the same error policy, and delete removeEncodingHints() from the serializer path.

Each slice passes the Lab gate and the A/B output_sha256 comparison, or reports the pages whose output changed.

Part of the simplification tracker #4368.


AI disclosure: drafted by Claude Opus (Anthropic) via Pi/Roadie from source analysis and a local libxml probe, for Chris Huber.

Dominant language
PHP
Stars
28
Forks
5
Avg merge
2h 59m
Merged PRs (30d)
942

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Automattic/blocks-engine

All issues in Automattic/blocks-engine

Similar issues

More PHP issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.