Tracing crashes with UnicodeEncodeError on surrogate escapes in hook arguments/results
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 85/100
Research direction
Start with _format_message in _tracing.py around line 38 and the hook result tracing call in _manager.py around line 506. Verify tracing with surrogate-containing hook arguments and results no longer raises UnicodeEncodeError, while structural labels remain readable and values are safely represented.
Written by the indexing model from the issue text.
Description
Problem
When pluggy tracing is enabled (e.g. pytest's --debug flag), hook arguments or return values containing surrogate escape characters (like \ud800) cause a UnicodeEncodeError crash. This is because _format_message in _tracing.py uses str() to format values, and the resulting string with literal surrogates cannot be written to most output targets.
Reported originally as pytest-dev/pytest#13750.
Analysis
_format_message currently formats two kinds of data identically using str():
- Structural labels passed as positional args — e.g.
"finish","-->", hook names — which are always safe ASCII strings - Python values — hook kwargs values (rendered via the
extradict) and hook return values — which may contain arbitrary data including surrogates
The fix needs to apply repr() only to the value positions, not to structural labels.
Intended Solution
-
In
_tracing.pyline 38 — use{value!r}for the extra/kwargs dict values:lines.append(f"{indent} {name}: {value!r}\n")This makes kwargs values in trace output show their type (e.g.
'lfplugin'instead oflfplugin,PosixPath('/foo')instead of/foo) and safely escapes surrogates. -
In
_manager.pyline 506 — userepr()on the hook result before passing it as a trace arg:hooktrace("finish", hook_name, "-->", repr(outcome.get_result()))This ensures the result value is safely formatted, while
"finish",hook_name, and"-->"remain plainstr()-formatted (via_format_message's existingmap(str, args)). -
Do NOT change the
content = " ".join(map(str, args))line in_format_message. Keepingstr()there preserves readable structural output without quoting labels. The caller (_manager.py) is responsible for pre-formatting any unsafe values withrepr().
This avoids the double-repr problem and keeps trace output readable:
finish pytest_runtest_call --> '\ud800' [hook]
config: <Config object at 0x...>
plugin_name: 'lfplugin'
Rather than the over-quoted version that blanket repr() in _format_message would produce:
'finish' 'pytest_runtest_call' '-->' "'\\ ud800'" [hook]
References
- pytest-dev/pytest#13750
- #627 (PR with discussion leading to this distilled approach)
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 160
- Avg merge
- 21h 4m
- Merged PRs (30d)
- 6
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from pytest-dev/pluggy
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
pytest-dev/pluggy#705 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 20/100
pytest-dev/pluggy#628 · 2 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
pytest-dev/pluggy#588 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
pytest-dev/pluggy#442 · 2 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
pytest-dev/pluggy#437 · 3 comments ·
All issues in pytest-dev/pluggy
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100