needs_separator_when_before omits CDC, so serialized tokens re-parse wrong

Open Beginner friendly
#434 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
rust
Domain
tooling

Research direction

Start in src/serializer.rs around line 519 and read needs_separator_when_before, especially the rows for Number, DelimHash, DelimAt, and DelimMinus. Verify the four CDC combinations against the issue's reproducers and existing serializer coverage. Done means each combination requests a separator and serialized tokens re-parse to the original token sequence.

Written by the indexing model from the issue text.

Description

TokenSerializationType::needs_separator_when_before returns false when a Number, DelimHash, DelimAt, or DelimMinus is followed by CDC. Concatenating the two serializations produces text that tokenizes into different tokens than the input.

Reproducer

cssparser 0.37.0, default features.

use cssparser::{Parser, ParserInput, ToCss};

fn main() {
    let mut pi = ParserInput::new("5 -->");
    let mut p = Parser::new(&mut pi);
    let a = p.next().unwrap().clone(); // Number { value: 5.0, int_value: Some(5) }
    let b = p.next().unwrap().clone(); // CDC

    // no separator requested
    assert!(!a
        .serialization_type()
        .needs_separator_when_before(b.serialization_type()));

    let mut s = String::new();
    a.to_css(&mut s).unwrap();
    b.to_css(&mut s).unwrap();
    assert_eq!(s, "5-->");
}

Observed vs expected

Observed: the call returns false and serialization yields 5-->. Re-parsing 5--> gives Dimension { value: 5.0, unit: "--" } followed by Delim('>').

Expected: true, which signals that an empty comment is needed so the output round-trips. The doc comment on needs_separator_when_before says it returns true if "an empty comment /**/ needs to be inserted between them so that they are not re-parsed as a single token". CSS Syntax Level 3 §Serialization requires the serialized form to round-trip.

Root cause

src/serializer.rs:519. The Ident and AtKeywordOrHash | Dimension rows list CDC. The rows covering Number, DelimHash, DelimAt, and DelimMinus do not.

Scope

Four pairs return false: Number then CDC gives 5-->, re-parsing as Dimension{5,"--"} + Delim('>'). DelimHash then CDC gives #-->, re-parsing as IDHash("--") + Delim('>'). DelimAt then CDC gives @-->, re-parsing as AtKeyword("--") + Delim('>'). DelimMinus then CDC gives --->, re-parsing as Ident("---") + Delim('>').

Any consumer using this API to re-emit CSS is affected. Stylo calls it for custom-property and var() substitution serialization. Present on main as of the 2026-07-21 commit.

Dominant language
Rust
Stars
869
Forks
152
Avg merge
15h 8m
Merged PRs (30d)
12

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from servo/rust-cssparser

All issues in servo/rust-cssparser

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.