🐛 Bug Report: LiteLLM instrumentation never ends the span when a call is interrupted (KeyboardInterrupt, asyncio cancellation)
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 80/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- observability-sre
Research direction
Start with _wrap_factory and _awrap_factory in opentelemetry/instrumentation/litellm/__init__.py, where spans are handled around completion and embedding calls. Reproduce the interruption with the supplied KeyboardInterrupt example, then add or update the package’s LiteLLM instrumentation tests to cover interrupted synchronous and asynchronous calls. Done means spans are ended and interruptions are recorded without changing the original exception behavior.
Written by the indexing model from the issue text.
Description
Which component is this bug for?
Traceloop SDK
📜 Description
Package: opentelemetry-instrumentation-litellm 0.62.4 (not in the component list, so I picked the closest option).
In the LiteLLM instrumentation, _wrap_factory and _awrap_factory (opentelemetry/instrumentation/litellm/__init__.py, lines 681 and 717 at 0.62.4) end the span only in except Exception (via _handle_exception) or after a successful return:
try:
response = wrapped(*args, **kwargs)
except Exception as error:
_handle_exception(span, metrics, error, model, kwargs)
raise
finally:
context_api.detach(token)
KeyboardInterrupt (Ctrl-C) and asyncio.CancelledError (a cancelled task, a timeout via asyncio.wait_for, a client disconnect in a web server) derive from BaseException, not Exception, so the span is never ended and never exported. This affects completion, acompletion, embedding and aembedding. Cancelling LLM calls is common in async servers, so these calls silently disappear from the traces; the open spans also stay referenced.
This is the same pattern as #4576 (other instrumentations), here in the LiteLLM package.
👟 Reproduction steps
pip install litellm opentelemetry-instrumentation-litellm opentelemetry-sdk httpx
import os
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True"
import httpx, litellm, openai
from opentelemetry.sdk.trace import SpanProcessor, TracerProvider
from opentelemetry.instrumentation.litellm import LiteLLMInstrumentor
class Count(SpanProcessor):
started = ended = 0
def on_start(self, span, parent_context=None): Count.started += 1
def on_end(self, span): Count.ended += 1
provider = TracerProvider()
provider.add_span_processor(Count())
LiteLLMInstrumentor().instrument(tracer_provider=provider)
def respond(request): # local stand-in: the user presses Ctrl-C while waiting for the server
raise KeyboardInterrupt
client = openai.OpenAI(api_key="x", max_retries=0, http_client=httpx.Client(transport=httpx.MockTransport(respond)))
for call in (lambda: litellm.completion(model="openai/gpt-4o-mini", messages=[{"role": "user", "content": "hi"}],
client=client, api_key="x"),
lambda: litellm.embedding(model="openai/text-embedding-3-small", input=["hi"], client=client, api_key="x")):
Count.started = Count.ended = 0
try:
call()
except KeyboardInterrupt:
pass
print("spans started", Count.started, "ended", Count.ended)
👍 Expected behavior
The span ends in every case, with the interruption recorded (e.g. error.type = KeyboardInterrupt / CancelledError, status ERROR), e.g. by catching BaseException in the wrappers (re-raising it unchanged) or ending the span in finally.
👎 Actual Behavior with Screenshots
F plain call interrupted by KeyboardInterrupt
app got: KeyboardInterrupt
spans started 1, ended 0
G embedding interrupted by KeyboardInterrupt
app got: KeyboardInterrupt
spans started 1, ended 0
The OpenInference LiteLLM instrumentation ends the span in the same situation.
🤖 Python Version
3.12
📃 Provide any additional context for the Bug.
opentelemetry-instrumentation-litellm 0.62.4 (the latest release), litellm 1.104.2. The same applies to acompletion / aembedding cancelled with asyncio.CancelledError (same except Exception in _awrap_factory).
👀 Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
None
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 2
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from traceloop/openllmetry
-
bug python
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4717 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: Incomplete cache token migration after #4243 breaks Bedrock and Groq instrumentationPossibly taken @Quantum-Enigma-dot claimed this 21 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4665 · 1 comment ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.reasoning_tokens on streamed chat completionsPossibly taken @jayfeng7 claimed this 23 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
traceloop/openllmetry#4660 · 1 comment ·
Maintainers usually reply within 1 day
-
Bug Report: Failed dataset deletes still remove local rows and columnsPossibly taken @inchang-ing claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
traceloop/openllmetry#4630 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: LiteLLM instrumentation does not record seed, stop, frequency_penalty, presence_penaltyPossibly taken @shantam77 claimed this 38 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
traceloop/openllmetry#4598 · 2 comments ·
Maintainers usually reply within 1 day
All issues in traceloop/openllmetry
Similar issues
-
Internal ErrorOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
frappe/frappe_docker#2071 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
The-Pocket/PocketFlow#161 ·
-
comp/desktop P2 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#157702 ·
Maintainers usually reply within 1 day
-
scibench_physics / scibench_chemistry: a correct answer scores 0 when the unit has a power of tenPossibly taken @BahaaMattar claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
EvolvingLMMs-Lab/lmms-eval#1699 ·
Maintainers usually reply within 1 day
-
project_inputs.html uses undefined file variable for dropdown IDs, lacks visual tag badges, and has empty delete labelsPossibly taken @Rucha0901 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
aboutcode-org/scancode.io#2354 ·
Maintainers usually reply within 1 day