[Bug]: try_parse_json_object turns backslash escapes into spaces when valid JSON comes in a code fence or after a sentence

Aperta Adatta ai principianti
  • Qualcosa non va?
#2,626 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
74/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai

Direzione di ricerca

Inizia da packages/graphrag/graphrag/query/llm/text_utils.py in try_parse_json_object, dove il ritaglio dal primo { all'ultimo } passa attraverso la pulizia delle barre rovesciate con .replace prima di qualsiasi controllo di validità. Esegui lo script di riproduzione nell'issue per confermare che line1\nline2 restituisce line1 nline2, e verifica che GlobalSearch._parse_search_response nel search.py della ricerca globale mostri lo stesso danno. È fatto quando il JSON valido ritagliato conserva i suoi escape, l'input non valido passa ancora per la vecchia pulizia e per json_repair, e un test di regressione copre le risposte con recinzioni e con prefisso.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Do you need to file an issue?
  • I have searched the existing issues and this bug is not already filed.
  • My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
  • I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.
Describe the bug

try_parse_json_object in packages/graphrag/graphrag/query/llm/text_utils.py first calls json.loads on the whole response. If the model wraps valid JSON in a markdown fence or puts a sentence before it, that call fails. The function then cuts out the text from the first { to the last } and runs a cleanup chain on it. One step of that chain is .replace("\\", " "). It runs before anything checks whether the cut-out text is already valid JSON.

So every escape inside a string value is broken, even though the cut-out text was valid:

  • \n becomes a space and the letter n. line1\nline2 comes back as line1 nline2.
  • \" becomes a space and a quote. He said \"hi\" today comes back as He said "hi " today.
  • \\ becomes two spaces. C:\\data\\x.txt comes back as C: data x.txt.
  • \t, \r and \u00e9 are broken the same way.

The same JSON without the fence parses correctly, because the first json.loads succeeds.

This function parses model output in three query paths:

  • the global search map step, GlobalSearch._parse_search_response (the description text of each key point)
  • rate_relevancy in dynamic community selection
  • DRIFT, DriftAction.search (the response text and follow_up_queries)

Map and DRIFT answers are markdown, so they often contain \n. A fenced map response is a real case: #1173 shows gemma2 on ollama returning the map response inside a ```json fence. Also, #2625 shows that response_format_json_object=True does not reach the model on main, and it reports that Qwen and Mistral on vLLM sometimes answer with prose around the JSON. I did not find a trace from OpenAI or Azure, so this mostly affects local and other non-JSON-mode providers.

Back in #801 a maintainer review asked for a json.loads check first, so valid JSON would skip the string cleanup (done in #834). That check only covers the whole response. It does not cover the object cut out of a fenced or prefixed response.

Steps to reproduce

No model is needed. With graphrag installed from main:

import json

from graphrag.query.llm.text_utils import try_parse_json_object
from graphrag.query.structured_search.global_search.search import GlobalSearch

BS = "\\"
raw = "```json\n" + '{"d": "line1' + BS + 'nline2"}' + "\n```"
expected = json.loads(raw[raw.index("{") : raw.rindex("}") + 1])
_, got = try_parse_json_object(raw, verbose=False)
print("expected:", repr(expected))
print("got:     ", repr(got))

map_raw = (
    "```json\n"
    + '{"points": [{"description": "first line' + BS + 'nsecond line, said '
    + BS + '"ok' + BS + '"", "score": 80}]}'
    + "\n```"
)
print(GlobalSearch._parse_search_response(None, map_raw))
Expected Behavior

When the cut-out object is already valid JSON, it should be returned as parsed. The cleanup and json_repair should only run when it is not valid.

Expected output:

expected: {'d': 'line1\nline2'}
got:      {'d': 'line1\nline2'}
[{'answer': 'first line\nsecond line, said "ok"', 'score': 80}]
GraphRAG Config Used
# Not needed. The reproduction calls the parser directly and uses no settings file.
Logs and screenshots

Output on main:

expected: {'d': 'line1\nline2'}
got:      {'d': 'line1 nline2'}
[{'answer': 'first line nsecond line, said  "ok "', 'score': 80}]
Additional Information
  • GraphRAG Version: 3.3.0 (main at 34b2b67)
  • Operating System: Windows 11
  • Python Version: 3.13
  • Related Issues: #1173 (fenced map response from gemma2 on ollama, fixed the regex then). #2625 (JSON mode is not sent to the model). Point 2 of #2625 also shows the backslash cleanup splitting single-quoted follow-up queries at \'. That input is not valid JSON, so the fix below does not change it.
  • I have a small fix with a regression test. After the cut-out step it tries json.loads and returns the result when it is a non-empty dict. Everything else still goes through the old cleanup path. I will open a PR that references this issue.

Validated with Claude Opus 5.5.

Lingua principale
Python
Stelle
36.6k
Fork
3.9k
Merge medio
4g 11h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/graphrag

Tutte le issue di microsoft/graphrag

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.