[Bug]: response_format_json_object=True is silently dropped, so JSON mode never reaches the model

Bắt đầu từ packages/graphrag-llm/graphrag_llm/completion/lite_llm_completion.py, nơi _base_completion và _base_completion_async kiểm tra "response_format" not in new_args trước khi bật chế độ JSON. Khóa này luôn tồn tại với giá trị None, vì vậy điều kiện nên kiểm tra None thay thế. Sau đó xem QueryState.add_all_follow_ups và find_incomplete_actions trong graphrag/query/structured_search/drift_search/state.py, các hàm này nên bỏ qua các follow-up không phải chuỗi. Hoàn thành khi script tái hiện cho thấy {'type': 'json_object'} được truyền vào LiteLLM, tái hiện 2 không còn ném AttributeError và các unit test bao phủ cả hai trường hợp.

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

Note on the provider checkbox: we run open-weight models via vLLM, but this bug does not depend on the provider. The reproduction below uses model_provider="openai" and never contacts a model.

Several query paths request JSON mode with response_format_json_object=True: the DRIFT local search actions (drift_search/search.py#L97), the global search map step (global_search/search.py#L97, #L240) and rate_relevancy (#L62). The flag never takes effect: the request sent to the model contains no response_format.

Root cause (graphrag_llm/completion/lite_llm_completion.py):

• completion and completion_async pop response_format (default None) and always pass it on explicitly (#L158, #L197).
• _base_completion and _base_completion_async only enable JSON mode if the key is absent (#L262, #L288):

  if json_object and "response_format" not in new_args:
  

Since the key is always present with the value None, this branch is never taken. LiteLLM then receives response_format=None and drops it.

Consequences in DRIFT (silent, only warnings are logged):

1. Without JSON mode, models often answer an action in plain prose. try_parse_json_object then returns {}, and the action yields no answer and no follow-ups ("No answer found for query").
2. Models also return non-strict JSON with single quotes. The repair in try_parse_json_object splits follow-up queries at escaped apostrophes and at the comma before a year, and the year becomes an int. Example: '...SBF\'s political dealings were mentioned in the October 7, 2023 TechCrunch digest?' becomes ["What specific details about SBF ", "s political dealings were mentioned in the October 7", 2023, "TechCrunch digest?'"].
3. QueryState.add_all_follow_ups logs a warning for non-string items but still adds them to the graph (state.py#L45-L53). find_incomplete_actions then raises AttributeError: 'int' object has no attribute 'is_complete' (#L57), and the whole DRIFT query fails.

Observed impact (vLLM 0.30.0, default DRIFT settings, MultiHop-RAG corpus): 5 to 25 % of DRIFT actions were answered in plain prose depending on the model (Qwen3.8-27B 5 to 14 %, Mistral-Small-3.2-24B 25 %). With Qwen3.8-27B, 3 to 5 % of DRIFT queries failed with the AttributeError even after two retries.

Steps to reproduce

1. JSON mode is dropped (no network access, no model needed):

import asyncio
import logging

import graphrag_llm.completion.lite_llm_completion as llc
from graphrag_llm.completion import create_completion
from graphrag_llm.config import ModelConfig

logging.getLogger("graphrag_llm").setLevel(logging.CRITICAL)  # hide the logged RuntimeError below

captured = {}


async def fake_acompletion(**kwargs):
    captured.update(kwargs)
    raise RuntimeError("stop before any network call")


llc.litellm.acompletion = fake_acompletion

model = create_completion(
    ModelConfig(model_provider="openai", model="gpt-4o-mini", api_key="not-used")
)


async def main():
    try:
        await model.completion_async(
            messages=[{"role": "user", "content": "Answer in JSON."}],
            response_format_json_object=True,
        )
    except RuntimeError:
        pass


asyncio.run(main())
print("response_format passed to litellm:", captured.get("response_format"))
# Actual:   None
# Expected: {'type': 'json_object'}

2. A non-string follow-up crashes DRIFT:

from graphrag.query.structured_search.drift_search.action import DriftAction
from graphrag.query.structured_search.drift_search.state import QueryState

state = QueryState()
parent = DriftAction(query="parent question")
state.add_action(parent)
state.add_all_follow_ups(parent, ["a valid follow-up question", 2023])
state.find_incomplete_actions()
# AttributeError: 'int' object has no attribute 'is_complete'

Expected Behavior

When response_format_json_object=True is passed and no explicit response_format is given, LiteLLM receives response_format={"type": "json_object"}. A follow-up query that is not a string should be skipped instead of crashing the DRIFT search.

GraphRAG Config Used

# Excerpt of settings.yaml. All other settings are GraphRAG 3.1.0 defaults.
# The JSON-mode bug does not depend on this config: reproduction 1 runs without any settings file.
completion_models:
  default_completion_model:
    model_provider: hosted_vllm   # bug also reproduced with model_provider: openai (reproduction 1)
    model: Qwen/Qwen3.8-27B       # also observed with mistralai/Mistral-Small-3.2-24B-Instruct-2506
    api_base: ${VLLM_API_BASE}
    api_key: ${GRAPHRAG_API_KEY}
    call_args:                    # query time
      temperature: 0
      top_p: 1
      max_tokens: 4000
    retry:
      type: exponential_backoff
      max_retries: 2

embedding_models:
  default_embedding_model:
    model_provider: hosted_vllm
    model: BAAI/bge-m3
    api_base: ${VLLM_EMBED_API_BASE}
    api_key: ${GRAPHRAG_API_KEY}

drift_search:                     # GraphRAG 3.1.0 defaults
  completion_model_id: default_completion_model
  embedding_model_id: default_embedding_model
  n_depth: 3
  drift_k_followups: 20
  primer_folds: 5
  concurrency: 32

Logs and screenshots

Reproduction 1 prints: response_format passed to litellm: None
Reproduction 2 raises: AttributeError: 'int' object has no attribute 'is_complete' (state.py, line 57)

Additional Information

• Versions: graphrag 3.1.0, graphrag-llm 3.1.0, litellm 1.86.2, Python 3.13. The same code is in v3.1.2, v3.2.0, v3.3.0 and on main (34b2b67).
• Suggested fix 1, in both _base_completion and _base_completion_async:
  if json_object and new_args.get("response_format") is None:
  With this change, reproduction 1 prints {'type': 'json_object'}.
• Suggested fix 2 (defensive): in QueryState.add_all_follow_ups, continue after the warning for non-string items. With fix 1 the trigger mostly disappears, but json_object mode does not constrain types, so a model can still return non-string items.
• I am happy to open a PR with both changes and unit tests.