logging: StructuredLogHandler writes invalid JSON when trace or span_id contains " or \

Open Beginner friendly
#19,099 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
68/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend

Research direction

The issue points to packages/google-cloud-logging/google/cloud/logging_v2/handlers/structured_log.py and handlers.py; start by reading how StructuredLogHandler formats and filters trace and span values. Reproduce the failure with the supplied filter() and format() example, then check the relevant handler tests. Done means formatted output parses as JSON and preserves trace and span strings containing quotes or backslashes.

Written by the indexing model from the issue text.

Description

Determine this is the right repository
  • I determined this is the correct repository in which to report this bug.
Summary of the issue

StructuredLogHandler writes invalid JSON when a trace or span id contains " or \. Its format string puts %(_trace_str)s and %(_span_id_str)s inside JSON quotes (structured_log.py#L33), and the filter copies the raw values (handlers.py#L111) without JSON escaping. The handler's own X-Cloud-Trace-Context parser accepts only [\w-]+ (_helpers.py#L195), so the default path is safe. Values set explicitly (extra={"trace": ...}, a logging filter, a framework integration) go into the line as-is, and Cloud Logging receives one broken text line instead of a structured entry.

Expected: valid JSON with both strings preserved. Actual: json.loads fails (Expecting ',' delimiter for the quote, Invalid \escape for the backslash).

API client name and version

google-cloud-logging 3.17.0 (also 3.12.1)

Reproduction steps: code

No client, credentials or network: only filter() and format() run.

"""Explicit trace/span strings must round-trip through structured JSON logging."""
import json
import logging
import sys
import traceback
from importlib.metadata import version

control = True
try:
    print("google-cloud-logging", version("google-cloud-logging"), flush=True)
    from google.cloud.logging_v2.handlers import StructuredLogHandler

    handler = StructuredLogHandler()

    def encode(trace_value, span_value):
        record = logging.LogRecord("repro", logging.INFO, "repro.py", 1, "hello", (), None)
        record.trace = trace_value
        record.span_id = span_value
        assert handler.filter(record)
        return handler.format(record)  # No emit(), client, credentials, or network.

    trace_key = "logging.googleapis.com/trace"
    span_key = "logging.googleapis.com/spanId"
    decoded = json.loads(encode("projects/p/traces/abc", "123"))
    assert decoded[trace_key] == "projects/p/traces/abc" and decoded[span_key] == "123"
    print("CONTROL PASSED: plain trace/span values round-trip through valid JSON")
    control = False
    failures = []
    for label, trace_value, span_value in (
        ("trace quote", 'projects/p/traces/a"b', "123"),
        ("span backslash", "projects/p/traces/abc", r"bad\span"),
    ):
        try:
            decoded = json.loads(encode(trace_value, span_value))
        except json.JSONDecodeError as error:
            failures.append(f"{label}: {error.msg}")
        else:
            if decoded[trace_key] != trace_value or decoded[span_key] != span_value:
                failures.append(f"{label}: value corrupted")
    handler.close()
    prefix = "BUG REPRODUCED: " if failures else "NOT REPRODUCED: "
    print(prefix + f"escaping failures={failures}; expected valid JSON preserving both strings")
except Exception:
    traceback.print_exc()
    if control:
        print("CONTROL FAILED")
    sys.exit(2 if control else 1)

Output:

google-cloud-logging 3.17.0
CONTROL PASSED: plain trace/span values round-trip through valid JSON
BUG REPRODUCED: escaping failures=["trace quote: Expecting ',' delimiter", 'span backslash: Invalid \\escape']; expected valid JSON preserving both strings
OS & version + platform

Linux

Python environment

Python 3.11

Additional context

Escaping these two fields like the message (json.dumps(value)[1:-1]) would fix it, or validating them against the same pattern the header parser uses.

Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
2d 20h
Merged PRs (30d)
140

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from googleapis/google-cloud-python

All issues in googleapis/google-cloud-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.