Responses API (`/v1/responses`) and Vertex AI: model access for a multi-turn benchmark

This is a design proposal, so start by deciding the approach rather than writing code. Read models/clients/factory.py (the _SUPPORTED_PROVIDER_TYPES check), models/clients/adapters/openai_compatible.py (where _ROUTE_CHAT is fixed to /chat/completions) and models/clients/parsing.py (where parse_assistant_message drops reasoning items). Done means a maintainer-agreed design covering routing, passed-through fields and replay of reasoning items.

Priority Level: Medium (Nice to have)

Is your feature request related to a problem? Please describe.

OpenAI models already work through the OpenAI-compatible adapter (Chat Completions). This is only about
OpenAI's other endpoint, /v1/responses, and only for multi-turn calls to reasoning models.

The benchmark below needs two things from Data Designer: the Responses API (this section), and Vertex AI
(see "Second gap: Vertex AI" below, folded in from #92).

Some evaluations send a model its own earlier reasoning back on the next turn, because models trained on
preserved thinking expect it. MindEval (Apache-2.0), a
multi-session counselling benchmark that runs on NeMo User Sim, does this for the model under test, and
User Sim now offers the same as an opt-in.

OpenAI's reasoning models return their reasoning only through the Responses API, as reasoning items
(encrypted when the call asks for include: ["reasoning.encrypted_content"] with store: false). Through
Chat Completions they return none. So through Data Designer, which reaches OpenAI-type endpoints with its
OpenAI-compatible adapter, these models can be called but their reasoning cannot be carried to the next
turn: they are evaluated without the context they would have in deployment, and their scores are not
comparable with runs that carry it.

Today MindEval works around this by calling those models through litellm's Responses bridge
(openai/responses/<model>, azure/responses/<deployment>) instead of Data Designer. Its README:

OpenAI reasoning models return it only through the Responses API: run them as
azure/responses/<deployment> (or openai/responses/<model>) with
--counselor_params '{"include": ["reasoning.encrypted_content"], "store": false}'.

Describe the solution you'd like

A way for a Data Designer model to use the Responses API and hand back what is needed to continue the
conversation, for example:

• a provider or endpoint option that sends a model's calls to /v1/responses (OpenAI and Azure OpenAI)
• request fields passed through (include, store, reasoning)
• the reasoning items returned on the response, and accepted again on the earlier assistant turns of the
  next request, so a caller can replay them

If there is already a way to do this in Data Designer, a pointer would be just as good, which is why
this is filed as a question as much as a request.

Describe alternatives you've considered

• Chat Completions only: works, but the reasoning is lost between turns (the problem above).
• litellm's Responses bridge outside Data Designer: what MindEval does now. It works, but those calls
  bypass Data Designer's model layer, so User Sim's accounting, retries and model configuration don't
  apply to them.
• A proxy in front of Data Designer (e.g. a LiteLLM proxy translating /chat/completions to
  /responses): gets the call through, but the reasoning items come back as an extra field that Data
  Designer's parser drops, and there is no way to send them back on the next turn.
• previous_response_id with store: true: keeps the reasoning on OpenAI's side instead of
  carrying it. Not usable where data must not be stored, and it ties each turn to one provider session.

Agent Investigation

Checked on main (2026-10-14) and in 0.9.4:

• models/clients/factory.py: _SUPPORTED_PROVIDER_TYPES = ("openai", "anthropic"); any other
  provider type raises. Plugins can't add a model client (plugin types: column generator, seed reader,
  processor).
• models/clients/adapters/openai_compatible.py: chat calls always post to _ROUTE_CHAT = "/chat/completions"; no other route for chat.
• models/clients/parsing.py: parse_assistant_message keeps content, reasoning_content (text),
  tool_calls and images; other message fields, such as reasoning items, are dropped.
• No existing issue found ("responses api", "encrypted_content", "/v1/responses").
• In MindEval (mindeval/llm.py), litellm returns the items as reasoning_items (or under
  provider_specific_fields) on the message; the benchmark stores them with the turn and sends them back
  on that turn next time (mindeval/session.py, CARRIED = ("reasoning_content", "thinking_blocks", "reasoning_items")).

Second gap: Vertex AI

The same benchmark uses Claude and Gemini on Vertex AI, today through litellm's vertex_ai/ provider,
outside Data Designer's model layer. What blocks it, as far as we can see in the code (main):

• Claude on Vertex (per Google Cloud's API docs) speaks the Anthropic Messages format, but at
  …/publishers/anthropic/models/{model}:rawPredict, with the model in the URL,
  anthropic_version: "vertex-2023-10-16" in the body, and a Google access token. The Anthropic adapter
  (models/clients/adapters/anthropic.py) posts to {endpoint}/v1/messages with x-api-key and the
  model in the body.
• Gemini on Vertex works through Vertex's OpenAI-compatible endpoint, but authenticates with a Google
  access token that expires after about an hour, while a provider's key is resolved once when the client
  is built (models/clients/factory.py, _resolve_api_key). A long run fails once the token expires.

Useful pieces: Google credentials (Application Default Credentials or a service account) with token
refresh during a run, and the Vertex URL and body shape for Claude.

Additional context

Related: User Sim's opt-in replay of the assistant's own reasoning (NVIDIA-NeMo/UserSim#31) currently
covers OpenAI-compatible endpoints that return reasoning_content; this would extend it to OpenAI's own
reasoning models.

• I've reviewed existing issues and the documentation
• This is a design proposal, not a "please build this" request