`CallsiteParameterAdder` use `additional_ignores` for foreign events
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
Research direction
Start by reading CallsiteParameterAdder.__call__ in src/structlog/processors.py and the linked open pull request #816. The issue describes a stack trace where additional_ignores does not affect foreign logging events; compare that path with the _from_structlog path. Done means foreign events use the configured ignores and the callsite points to the user's code.
Written by the indexing model from the issue text.
Description
I was configuring some stdlib logs (eg requests) and I want the log to emit with the callsite from my code not the lib, and so I added additional_ignores=["urllib3", "requests"] alas it doesn't do it if not _from_structlog
https://github.com/hynek/structlog/blob/main/src/structlog/processors.py#L906-L928
def __call__(
self, logger: logging.Logger, name: str, event_dict: EventDict
) -> EventDict:
record: logging.LogRecord | None = event_dict.get("_record")
from_structlog: bool = event_dict.get("_from_structlog", False)
# If the event dictionary has a record, but it comes from structlog,
# then the callsite parameters of the record will not be correct.
if record is not None and not from_structlog:
for mapping in self._record_mappings:
event_dict[mapping.event_dict_key] = record.__dict__[
mapping.record_attribute
]
return event_dict
frame, module = _find_first_app_frame_and_name(
additional_ignores=self._additional_ignores
)
for parameter, handler in self._active_handlers:
event_dict[parameter.value] = handler(module, frame)
return event_dict
To remedy this I just did my own subclass
class CallsiteParameterAdder(_CallsiteParameterAdder):
_all_parameters: ClassVar[set[CallsiteParameter]] = set(CallsiteParameter)
def __init__(
self,
parameters: Collection[CallsiteParameter] = _all_parameters,
additional_ignores: list[str] | None = None,
) -> None:
additional_ignores = additional_ignores or []
super().__init__(parameters, [__name__, *additional_ignores])
def __call__(self, logger: logging.Logger, name: str, event_dict: EventDict) -> EventDict:
frame, module = _find_first_app_frame_and_name(additional_ignores=self._additional_ignores)
for parameter, handler in self._active_handlers:
event_dict[parameter.value] = handler(module, frame)
return event_dict
I am wondering if the comment If the event dictionary has a record, but it comes from structlog, then the callsite parameters of the record will not be correct. in the source is still true, it may be down to how I configure structlog in services but it isn't an issue.
eg here is the stack in structlog.processors.CallsiteParameterAdder
File "/temp.py", line 16, in <module>
requests.post("https://google.com")
File "/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/.venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/.venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/.venv/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
File "/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 791, in urlopen
response = self._make_request(
File "/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 547, in _make_request
log.debug(
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1527, in debug
self._log(DEBUG, msg, args, **kwargs)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1684, in _log
self.handle(record)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1700, in handle
self.callHandlers(record)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
hdlr.handle(record)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1028, in handle
self.emit(record)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1160, in emit
msg = self.format(record)
File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 999, in format
return fmt.format(record)
File "/.venv/lib/python3.12/site-packages/structlog/stdlib.py", line 1160, in format
ed = p(logger, meth_name, ed) # type: ignore[arg-type]
File "/.venv/lib/python3.12/site-packages/structlog/processors.py", line 896, in __call__
traceback.print_stack()
in condensed form you can see how just ignoring logging + additional_ignores suffices
temp.py
requests
urllib3
logging
structlog
- Dominant language
- Python
- Stars
- 5k
- Forks
- 302
- Avg merge
- 17m
- Merged PRs (30d)
- 4
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from hynek/structlog
-
bind_contextvars is not thread-safe?Possibly taken @nosmile99 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Transform examples in docs into included source filesPossibly taken @alexchen-sys claimed this 99 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
waiting for feedback
Difficulty 3/5 1-2 days Newbie friendliness 48/100
Similar issues
-
type-bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
push_gh-pages fails on master branch on forksPossibly taken A pull request linked to this issue is open or already merged. Opengood first issue infrastructure
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
kuznia-rdzeni/coreblocks#1135 ·
-
good first issue server
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
documentation priority: high
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
a-lex-s/mealie-telegram-bot#54 ·
Maintainers usually reply within 1 day