Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

v2: t:utf8toUnicode drops the last hex digit of code points above U+FFFF (regression since 2.9.9)

Open Beginner friendly
  • Is something wrong?
#3,684 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
apache, c
Domain
security

Research direction

The bug is in utf8_unicode_inplace_ex() in apache2/msc_util.c, in the branch that calls sprintf(data, "%%u%04x", d) and then does data += 6. Start there: the pointer advance should follow the length sprintf() returns, and bytes_left should track the remaining input. Add a four-byte sequence case to tests/tfn/utf8toUnicode.t. Done when U+1F600 transforms to %u1f600 and the existing BMP cases still pass.

Written by the indexing model from the issue text.

Description

Summary

In ModSecurity v2 (2.9.9 through 2.9.15, and current v2/master), t:utf8toUnicode emits only four of the five hex digits for any code point above U+FFFF (supplementary planes: emoji, CJK extensions, mathematical alphanumerics, ...). The last digit is overwritten by whatever the function writes next, or by the terminating NUL when the character is the last one in the value.

Related to #1211 (ambiguous output of this transformation), but this is a separate, newer defect: #1211 describes the 4-digit vs. 6-digit ambiguity by design; this issue is a plain truncation introduced by the refactor in #3284 (commit 907d61ad, fix for #3129), first released in 2.9.9. libModSecurity v3 is not affected (its port still writes the digits one by one), although it shares the #1211 ambiguity.

Reproduction

Output of utf8_unicode_inplace_ex() from apache2/msc_util.c at v2.9.15, compiled standalone:

Input Output Expected
U+1F600 (f0 9f 98 80) %u1f60 %u1f600
U+1F600 followed by A %u1f60A %u1f600A
a U+1F600 U+00E5 a%u1f60%u00e5 a%u1f600%u00e5
U+20AC (e2 82 ac) %u20ac %u20ac (BMP characters are fine)
Cause
if (unicode_len > 0) {
    i += unicode_len;
    sprintf(data, "%%u%04x", d);
    data += 6;          /* "%u" + 4 digits, but d > 0xFFFF prints 5 digits */
    *changed = 1;
}

Before 907d61ad the four-byte branch wrote the digits individually, so 5-digit code points were preserved.

Two adjacent problems in the same function
  • bytes_left is initialised to input_len and never decremented, so the "check we have at least N bytes" guards compare against the total length instead of the remaining length. A truncated multibyte sequence at the end of the value reads past input_len. In practice the read stops at the NUL that terminates every v2 variable value, so it is bounded to one byte, but the guard is wrong.
  • The range check only rejects lead bytes >= 0xF5; an 0xF4 lead byte with a second byte >= 0x90 encodes U+110000 and above, which is outside Unicode, and is currently accepted.
Impact

Correctness only, as far as I can tell: the output buffer is sized at ten bytes per input byte and a 4-byte sequence writes at most eight, so nothing is written outside the allocation, and the transformed string is always a superset of the ASCII input, so I could not construct a rule bypass with it. Rules that match %uXXXXX sequences for supplementary-plane characters do not work, and the transformed value is wrong in the audit log.

Proposed fix

Advance data by the value sprintf() returns instead of a constant 6, decrement bytes_left (or compare against input_len - i), reject code points above U+10FFFF, and add a four-byte sequence to tests/tfn/utf8toUnicode.t, which currently has none. PR to follow.

Dominant language
C++
Stars
9.8k
Forks
1.8k
Avg merge
1d 2h
Merged PRs (30d)
17

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from owasp-modsecurity/ModSecurity

All issues in owasp-modsecurity/ModSecurity

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.