🐛 Bug Report: Qdrant async spans end before the operation runs
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 4/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- python
- Domain
- observability
Research direction
Start in the Qdrant instrumentation code where _wrap is registered for both client classes, then inspect the actual methods on AsyncQdrantClient to see which are coroutines. Reproduce with the supplied script using AsyncQdrantClient.upsert and httpx.MockTransport; done means the awaited span covers the delay, parents the transport span, and records ERROR plus an exception event for HTTP 500. A linked pull request is already open.
Written by the indexing model from the issue text.
Description
Which component is this bug for?
Qdrant Instrumentation
📜 Description
Awaited AsyncQdrantClient operations produce spans that end before the operation executes. This makes request durations nearly zero, loses exceptions raised during the await, and attaches spans created inside the request to the caller instead of the Qdrant span.
The instrumentor registers the synchronous _wrap callback for both client classes. For a coroutine method, wrapped(*args, **kwargs) returns the coroutine immediately; the wrapper marks that truthy object as successful and exits the span context. wrapt invokes the supplied callback directly; it does not automatically await the coroutine or extend the context manager's lifetime.
👟 Reproduction steps
- Use
opentelemetry-instrumentation-qdrant0.62.4 andqdrant-client1.11.3. - Start a local Jaeger instance accepting OTLP/HTTP on
http://127.0.0.1:4318/v1/traces. - Save the script below as
reproduce.py, then runpython reproduce.py before. Thebeforeargument only labels the exported traces; run against the unchanged instrumentation. - Inspect the success and error traces in Jaeger, or the exported
evidence/before-spans.json.
The script uses the real AsyncQdrantClient.upsert method with httpx.MockTransport: it waits 250 ms, then returns either HTTP 200 or HTTP 500. No Qdrant server or credentials are needed.
Standalone reproducer
import argparse
import asyncio
import importlib.metadata as metadata
import json
from pathlib import Path
import httpx
from opentelemetry.instrumentation.qdrant import QdrantInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from qdrant_client import AsyncQdrantClient, models
from qdrant_client.http.exceptions import UnexpectedResponse
parser = argparse.ArgumentParser()
parser.add_argument('phase', choices=['before', 'after'])
parser.add_argument('--output-dir', type=Path, default=Path(__file__).parent / 'evidence')
args = parser.parse_args()
output = args.output_dir
output.mkdir(parents=True, exist_ok=True)
provider = TracerProvider(resource=Resource.create({
'service.name': f'qdrant-async-spans-{args.phase}',
'demo.transport': 'httpx.MockTransport; synthetic 250 ms latency',
}))
memory = InMemorySpanExporter()
provider.add_span_processor(SimpleSpanProcessor(memory))
provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint='http://127.0.0.1:4318/v1/traces')))
tracer = provider.get_tracer('qdrant-async-spans-reproduction')
QdrantInstrumentor().instrument(tracer_provider=provider)
async def scenario(fail):
async def handler(request):
with tracer.start_as_current_span('HTTP transport (simulated 250 ms)'):
await asyncio.sleep(0.25)
if fail:
return httpx.Response(500, json={'status': {'error': 'controlled demo failure'}})
return httpx.Response(200, json={
'result': {'operation_id': 1, 'status': 'completed'}, 'status': 'ok', 'time': 0.25,
})
client = AsyncQdrantClient(url='http://localhost:6333', transport=httpx.MockTransport(handler))
try:
try:
with tracer.start_as_current_span(f'Async upsert: {"error" if fail else "success"}'):
await client.upsert(collection_name='demo_collection', points=[models.PointStruct(id=1, vector=[0.1, 0.2])])
except UnexpectedResponse:
pass
finally:
await client.close()
async def main():
await scenario(False)
await scenario(True)
try:
asyncio.run(main())
provider.force_flush()
spans = [json.loads(span.to_json()) for span in memory.get_finished_spans()]
(output / f'{args.phase}-spans.json').write_text(json.dumps(spans, indent=2) + '\n')
summaries = []
for span in memory.get_finished_spans():
summaries.append({
'name': span.name, 'trace_id': f'{span.context.trace_id:032x}',
'span_id': f'{span.context.span_id:016x}',
'parent_id': f'{span.parent.span_id:016x}' if span.parent else None,
'duration_ms': round((span.end_time-span.start_time)/1e6, 3),
'status': span.status.status_code.name,
})
print(json.dumps({'phase': args.phase, 'versions': {n: metadata.version(n) for n in
['qdrant-client', 'wrapt', 'opentelemetry-sdk']}, 'spans': summaries}, indent=2))
(output / f'{args.phase}-summary.json').write_text(json.dumps(summaries, indent=2) + '\n')
finally:
QdrantInstrumentor().uninstrument()
provider.shutdown()
👍 Expected behavior
The Qdrant span stays open and current while the operation is awaited. Its duration covers the request, transport spans are its children, and an exception raised during the await sets ERROR and records an exception event.
Calling an async method without awaiting it should not export a completed successful operation span.
👎 Actual Behavior with Screenshots
In the reproduction:
- Success: the Qdrant span lasts 15 microseconds while the simulated transport takes about 251 ms. The transport span is its sibling.
- HTTP 500: the Qdrant span lasts 14 microseconds and remains OK with no exception event, even though the awaited operation raises
UnexpectedResponse. - Merely creating the coroutine exports a completed OK span before any request runs.
Jaeger screenshots
Captured from Jaeger 2.6.0 with the real client and instrumentor, using httpx.MockTransport with a controlled 250 ms delay. These show instrumentation lifecycle behavior, not live Qdrant server latency.
Success, before: the Qdrant span ends after 15 microseconds; the transport is a sibling.
Success, with the proposed fix: the Qdrant span lasts 262.125 ms and parents the transport span.
HTTP 500, before: the Qdrant span ends after 14 microseconds with status OK and no exception event.
HTTP 500, with the proposed fix: the Qdrant span lasts 261.740 ms, has status ERROR, and records one exception event.
🤖 Python Version
3.11.9 on macOS arm64; regression checks also run on Python 3.10 and 3.12.
📃 Provide any additional context for the Bug.
Reproduced against upstream commit 8d9c93f37b352131f5871dcb6ecf5e79c19aeb40, instrumentation 0.62.4, Qdrant client 1.11.3, OpenTelemetry SDK 1.45.1, and wrapt 2.5.1. Also reproduced with wrapt 1.17.3.
An async wrapper should await the operation inside the span context. Wrapper selection needs to inspect the actual method: AsyncQdrantClient also exposes synchronous upload methods. This is separate from missing method coverage and compatibility with newer Qdrant client releases.
👀 Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
Yes I am willing to submit a PR!
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 2
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from traceloop/openllmetry
-
bug python
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4717 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: Incomplete cache token migration after #4243 breaks Bedrock and Groq instrumentationPossibly taken @Quantum-Enigma-dot claimed this 21 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4665 · 1 comment ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.reasoning_tokens on streamed chat completionsPossibly taken @jayfeng7 claimed this 23 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
traceloop/openllmetry#4660 · 1 comment ·
Maintainers usually reply within 1 day
-
Bug Report: Failed dataset deletes still remove local rows and columnsPossibly taken @inchang-ing claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
traceloop/openllmetry#4630 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: LiteLLM instrumentation does not record seed, stop, frequency_penalty, presence_penaltyPossibly taken @shantam77 claimed this 38 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
traceloop/openllmetry#4598 · 2 comments ·
Maintainers usually reply within 1 day
All issues in traceloop/openllmetry
Similar issues
-
opensubtitlescom: provider rejects pt-PT even though its own converter maps itPossibly taken @paulo-0308 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Diaoul/subliminal#1405 ·
-
Internal ErrorOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
frappe/frappe_docker#2071 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
The-Pocket/PocketFlow#161 ·
-
comp/desktop P2 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#157702 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
zai-org/zcode-plugins#202 ·
Maintainers usually reply within 1 day