Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

[bot] Google GenAI: streaming responses drop `url_context_metadata` that non-streaming responses preserve

Ouverte Adaptée aux débutants
  • Quelque chose ne va pas ?
#774 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
2/5
Temps estimé
1-3 heures
Accessibilité débutants
86/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
python
Domaine
backend, testing

Piste de recherche

Commencez dans py/src/braintrust/integrations/google_genai/tracing.py, au niveau de _aggregate_generate_content_chunks(), puis comparez la gestion du champ candidate avec les tests existants des métadonnées de grounding dans py/src/braintrust/integrations/google_genai/test_google_genai.py. Ajoutez une couverture pour les métadonnées url_context en streaming dans les chemins synchrone et asynchrone, et vérifiez que les métadonnées sont présentes dans la sortie du span obtenu.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

<!-- provider-gap-audit: google-genai-streaming-url-context-metadata -->

Summary

When the Gemini url_context tool is used with generate_content_stream() / agenerate_content_stream(), the per-URL retrieval metadata (candidate.url_context_metadata) that Google's SDK returns is silently dropped from the Braintrust span output. The equivalent metadata for the google_search tool (candidate.grounding_metadata) is preserved in the exact same code path — so this is a fidelity gap between two structurally-analogous tool result types within the same function, not a "feature never built" gap.

Non-streaming generate_content() calls do not have this problem: the raw GenerateContentResponse object (including url_context_metadata) is logged as-is, so nothing is lost there.

What is missing

_aggregate_generate_content_chunks() in py/src/braintrust/integrations/google_genai/tracing.py (used by both the sync and async streaming wrappers) manually reconstructs a candidate_dict from the accumulated chunks, copying over only an explicit allowlist of candidate fields:

candidate_dict = {"content": {"parts": parts, "role": "model"}}

if hasattr(candidate, "finish_reason"):
    candidate_dict["finish_reason"] = candidate.finish_reason
if hasattr(candidate, "safety_ratings"):
    candidate_dict["safety_ratings"] = candidate.safety_ratings
if hasattr(candidate, "grounding_metadata") and candidate.grounding_metadata:
    candidate_dict["grounding_metadata"] = candidate.grounding_metadata

(py/src/braintrust/integrations/google_genai/tracing.py:695-709)

candidate.url_context_metadata — the field Google's own docs say to inspect to see "which URLs the model retrieved" when the url_context tool is enabled — is never copied into candidate_dict, so it never reaches the logged span output for streaming calls. Any user who calls client.models.generate_content_stream(..., config=GenerateContentConfig(tools=[{"url_context": {}}])) gets a span with no record of which URLs were actually fetched, even though the same call via generate_content() (non-streaming) would show it.

This is the same class of field (candidate.<x>_metadata describing what a built-in tool did) as grounding_metadata, which is explicitly captured here and has dedicated test coverage (test_google_search_grounding / test_google_search_grounding_async in test_google_genai.py). There is no equivalent test for url_context, and a full-file grep for url_context or code_execution in test_google_genai.py returns zero matches — confirming there is no regression coverage that would have caught this gap.

Note: the separate _TOOL_CALL_TYPES/_TOOL_RESULT_TYPES constants and interaction-tool-span logic elsewhere in the same file (tracing.py:54-69, :877-969) do already generically recognize url_context_call/url_context_result and code_execution_call/code_execution_result — that mechanism belongs to the newer content-item/"interactions" API surface and is unrelated to the classic generate_content_stream() candidate-based aggregation described above, which is the specific path where the metadata is lost.

Braintrust docs status

not_found — https://www.braintrust.dev/docs/integrations/ai-providers/google-genai (and the general https://www.braintrust.dev/docs/guides/tracing) do not document url_context tool support or grounding/citation-style metadata capture at all, streaming or otherwise.

Upstream sources

Local repo files inspected

  • py/src/braintrust/integrations/google_genai/tracing.py:
    • _aggregate_generate_content_chunks() (~lines 641-722) — builds candidate_dict for streaming span output; copies finish_reason, safety_ratings, grounding_metadata but not url_context_metadata
    • _gc_process_result() (~lines 573-581) — non-streaming path; returns the raw GenerateContentResponse, so no loss there
    • _TOOL_CALL_TYPES / _TOOL_RESULT_TYPES (~lines 54-69) and the interaction-tool-span logic (~lines 877-969) — confirmed this is a separate code path (content-item/interactions API) unrelated to the candidate-based streaming aggregation gap above
  • py/src/braintrust/integrations/google_genai/test_google_genai.py:
    • test_google_search_grounding / test_google_search_grounding_async (~lines 1451, 1557) and _assert_grounding_metadata (~line 1411) — dedicated grounding-metadata test exists for google_search only
    • Full-file grep for url_context and code_execution — zero matches, confirming no test coverage for either tool type
Langage dominant
Python
Étoiles
19
Forks
17
Merge moyen
1 j 5 h
PR mergées (30 j)
61

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de braintrustdata/braintrust-sdk-python

Toutes les issues de braintrustdata/braintrust-sdk-python

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.