Per-field truncation / summarization for large nested log fields
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start by reading the existing processors in structlog.processors and their tests to understand the processor API and conventions. The issue proposes field-specific string, list, and nested-dictionary limits but does not identify an entry point or define behavior for edge cases; clarify the API and expected summaries with maintainers before implementation. Done means a built-in processor handles the requested per-field limits and has tests covering its behavior.
Written by the indexing model from the issue text.
Description
Summary
It would be useful if structlog provided a built-in processor for per-field truncation / summarization of large values, especially nested dicts and lists.
Problem
Some fields in structured logs are much larger than others, for example:
- request / response payloads
- nested kwargs
- prompt text
- model outputs
These can overwhelm otherwise readable logs. Global string truncation is not enough; what is often needed is field-aware summarization.
Desired behavior
Something like a built-in processor where selected fields can be limited independently, for example:
- truncate long strings for
prompt - limit list length for
messages - limit depth / keys for nested dicts like
payload
Example
log.debug(
"completion.request",
prompt=very_large_prompt,
messages=large_messages,
extra_body=large_nested_dict,
)
Possible API shape:
structlog.processors.truncate_fields(
{
"prompt": {"max_length": 300},
"messages": {"max_items": 2, "max_string_length": 200},
"extra_body": {"max_depth": 1, "max_keys": 10},
}
)
Why
This seems like a common structured-logging need: preserve useful context while keeping summary/debug logs readable without writing a custom processor every time.
Would maintainers be open to something like this?
- Dominant language
- Python
- Stars
- 5k
- Forks
- 302
- Avg merge
- 17m
- Merged PRs (30d)
- 4
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from hynek/structlog
-
bind_contextvars is not thread-safe?Possibly taken @nosmile99 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Transform examples in docs into included source filesPossibly taken @alexchen-sys claimed this 98 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
waiting for feedback
Difficulty 3/5 1-2 days Newbie friendliness 48/100
-
`CallsiteParameterAdder` use `additional_ignores` for foreign eventsPossibly taken @hynek claimed this 147 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 25/100
Similar issues
-
follow-up
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Download helper still uses huggingface_hub arguments removed in 1.0: USE_SYMLINKS is silently ignored and every download ends with a WinError 3 linePossibly taken @li-lizhe claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
IPv6 node addresses produce an invalid request URLPossibly taken @RaphaelFakhri claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
typesense/typesense-python#154 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
elastic/elasticsearch-py#3592 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
developmentseed/titiler-cmr#250 · 1 comment ·
Maintainers usually reply within 1 day