Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Bug] MCP tool results containing image content are sent to the model as Base64 text instead of as images

Aperta
  • Qualcosa non va?
#465 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
java
Ambito
desktop, devtools

Direzione di ricerca

Start by tracing how MCP tool results from the stdio transport are converted into the model request, then compare that path with manually attached images and the VS Code behavior described here. The fix is complete when image content is forwarded as an image, and models without image support receive a clear warning instead of Base64 text.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

OS: Red Hat Enterprise Linux 9.6 (Plow)
Eclipse Version: Eclipse Platform 4.38.0 (v20260708-0722)
Plugin Version: GitHub Copilot for Eclipse 0.20.0.202607100822
Model: GPT-6 Astra (Medium), in Agent mode
MCP transport: stdio (local server built with FastMCP)

When an MCP tool returns an image in its result ({"type": "image", "mimeType": "image/jpeg", "data": ""}), Copilot for Eclipse sends it to the model as a Base64 text string, not as an image. The model can't see the picture, so it says it received "Base64-encoded JPEG text rather than a viewable image".

The same MCP server, tool and image work in VS Code Copilot Chat, where the model sees and describes the screenshot. Manually attaching the same JPEG file in Copilot for Eclipse chat also works. So, the model can handle images, and the problem appears to be in how the plugin forwards images from MCP tool results.

To Reproduce
FastMCP Example:

from fastmcp import FastMCP
from fastmcp.utilities.types import Image

mcp = FastMCP("image-test")

@mcp.tool
def get_image() -> Image:
    """Return a test image."""
    return Image(path="/path/to/any.jpg")

if __name__ == "__main__":
    mcp.run()

Register the server in Copilot for Eclipse's MCP settings (stdio transport).
Open Copilot Chat in Agent mode and pick a vision-capable model.
Ask: "Call get_image and describe what is in the image."
The tool runs successfully, but the model replies that it received Base64 text and can't view the image.

Expected behavior
Image content from MCP tool results should be sent to the model as images, as VS Code Copilot Chat does and as happens with images attached manually in Copilot for Eclipse. The model should then be able to describe the image.

If the selected model doesn't support images, the plugin should show a clear warning rather than sending raw Base64 text.

LLM prompt and output:
Prompt: use the screenshot tool to describe the image. DO NOT use any other context or information to guess what is on the screenshot.
Copilot:
The screenshot was returned as Base64-encoded JPEG data, not a viewable image. I can’t reliably describe its contents and won’t guess from other context.

Please attach the screenshot directly as an image or use a model/interface that can render the returned image.

The point of the screenshot tool is to screenshot the IDE and automatically look at the image. It is cumbersome to take a screenshot, add it to the project (or drag it to the document viewer in eclipse), and then manually add it to chat.

Lingua principale
Java
Stelle
125
Fork
63
Merge medio
1g 3h
PR unite (30g)
5

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/copilot-for-eclipse

Tutte le issue di microsoft/copilot-for-eclipse

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.