Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[FR]: Testing agent should handle AppFunctionUriGrant results and pass returned images to the model

Offen
  • Stimmt etwas nicht?
#63 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
android, kotlin
Bereich
ai, mobile

Rechercherichtung

Start with AgentOrchestrator.handleLlmResponse around line 301 and trace how function results are converted into chat content; compare the existing imageUri handling with AppFunctionUriGrant. Then inspect GeminiProviderImpl to see how the next model turn is built. Done means granted image URIs reach Gemini as image input and the returned image is handled in chat as described.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Description

A function can return an image the standard way, as an AppFunctionUriGrant (a content URI plus FLAG_GRANT_READ_URI_PERMISSION). The Testing Agent doesn't recognize this:

  1. Not shown: AgentOrchestrator.handleLlmResponse (around line 301) only picks up images from a result field named imageUri, the agent's own convention from generateImage (see #12). A standard AppFunctionUriGrant result is passed to the model as plain text, and no image appears in the chat.
  2. Not seen by the model: even with imageUri, the image is only attached to the chat message for the user. GeminiProviderImpl never sends image content to the model, so a function's returned image never reaches the LLM.
Why it matters

Visual apps rely on returning renders so the agent can check its own work. For example, a design app can expose renderDesign(): AppFunctionUriGrant and instruct the agent to render after changes and correct what it sees. With the Testing Agent, the model works blind, so this loop can't be tested at all, even though it is exactly the kind of flow a system agent would need to support.

Suggested behavior
  • Recognize AppFunctionUriGrant (or any result whose URI points to an image/* type) as an image result, alongside the imageUri convention.
  • Read it through the granted permission and attach it to the next model turn as image input (Gemini inline_data), so the model can see it.
  • Optionally show it inline in the chat, as #12 did for imageUri.
Environment
  • Testing Agent at a3a2a89 (main, Oct 23, 2026)
  • androidx.appfunctions 1.0.0-beta01
  • Android 17 (API 37) emulator

Related: #12 (inline image URIs in the chat), #62.

Vorherrschende Sprache
Kotlin
Sterne
210
Forks
45
Ø Merge
12 Std. 24 Min.
Gemergte PRs (30 T.)
3

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus android/appfunctions

Alle Issues in android/appfunctions

Ähnliche Issues

Weitere Issues zu Kotlin

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.