langchain-ai/langchain
View on GitHub`llm.ainvoke` is traced differently than `llm.invoke` (won't be traced)
Open
#30,870 opened on Apr 16, 2025
bugcoreexternalhelp wanted
Description
Checked other resources
- I added a very descriptive title to this issue.
- I used the GitHub search to find a similar question and didn't find it.
- I am sure that this is a bug in LangChain rather than my code.
- The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
- I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.
Example Code
We have a custom tracer implemented in python derived from AsyncBaseTracer. We noticed a diferent tracer behaviour when using llm.ainvoke vs llm.invoke
def some_node_in_graph():
tools = [add]
llm_with_tools = ChatAnthropic(api_key=get_anthropic_api_key(), model="claude-3-opus-20240229")().bind_tools(tools)
messages = [
SystemMessage(content="You are a helpfull asistant, please add two random numbers"),
HumanMessage(content=state.message),
]
response_message = llm_with_tools.invoke(messages) # this won't be traced
response_message = await llm_with_tools.ainvoke(messages) # this will work with no problems
# tracer
class AsyncCustomTracer(AsyncBaseTracer):
async def _start_trace(self, run: Run) -> None:
await super()._start_trace(run)
await self._persist_run(run) # our persist run will make an http post
async def _end_trace(self, run: Run) -> None:
await super()._end_trace(run)
await self._persist_run(run)
Error Message and Stack Trace (if applicable)
No response
Description
I've narrowed it down to https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/callbacks/manager.py#L275
- ainvoke will go to
except NotImplementedError as e: if event_name == "on_chat_model_start"and will work perfectly with no issues - invoke will raise
NotImplementedErrorand not be caught by this handler and will print an error log
Error in callback coroutine: NotImplementedError('Chat model tracing is not supported in for original format.')
Error in callback coroutine: TracerException('No indexed run ID 507c24d0-9f28-4737-aa66-551911a7b22c.')
System Info
Package Information
-------------------
> langchain_core: 0.3.51
> langchain: 0.3.23
> langchain_community: 0.3.21
> langsmith: 0.3.18
> langchain_anthropic: 0.3.10
> langchain_openai: 0.3.9
> langchain_text_splitters: 0.3.8
> langgraph_sdk: 0.1.59