Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Responses API (`/v1/responses`) and Vertex AI: model access for a multi-turn benchmark

Abierto
  • ¿Algo no está bien?
#1,033 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
api

Línea de trabajo

This is a design proposal, so start by deciding the approach rather than writing code. Read models/clients/factory.py (the _SUPPORTED_PROVIDER_TYPES check), models/clients/adapters/openai_compatible.py (where _ROUTE_CHAT is fixed to /chat/completions) and models/clients/parsing.py (where parse_assistant_message drops reasoning items). Done means a maintainer-agreed design covering routing, passed-through fields and replay of reasoning items.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Priority Level: Medium (Nice to have)

Is your feature request related to a problem? Please describe.

OpenAI models already work through the OpenAI-compatible adapter (Chat Completions). This is only about
OpenAI's other endpoint, /v1/responses, and only for multi-turn calls to reasoning models.

The benchmark below needs two things from Data Designer: the Responses API (this section), and Vertex AI
(see "Second gap: Vertex AI" below, folded in from #92).

Some evaluations send a model its own earlier reasoning back on the next turn, because models trained on
preserved thinking expect it. MindEval (Apache-2.0), a
multi-session counselling benchmark that runs on NeMo User Sim, does this for the model under test, and
User Sim now offers the same as an opt-in.

OpenAI's reasoning models return their reasoning only through the Responses API, as reasoning items
(encrypted when the call asks for include: ["reasoning.encrypted_content"] with store: false). Through
Chat Completions they return none. So through Data Designer, which reaches OpenAI-type endpoints with its
OpenAI-compatible adapter, these models can be called but their reasoning cannot be carried to the next
turn: they are evaluated without the context they would have in deployment, and their scores are not
comparable with runs that carry it.

Today MindEval works around this by calling those models through litellm's Responses bridge
(openai/responses/<model>, azure/responses/<deployment>) instead of Data Designer. Its README:

OpenAI reasoning models return it only through the Responses API: run them as
azure/responses/<deployment> (or openai/responses/<model>) with
--counselor_params '{"include": ["reasoning.encrypted_content"], "store": false}'.

Describe the solution you'd like

A way for a Data Designer model to use the Responses API and hand back what is needed to continue the
conversation, for example:

  • a provider or endpoint option that sends a model's calls to /v1/responses (OpenAI and Azure OpenAI)
  • request fields passed through (include, store, reasoning)
  • the reasoning items returned on the response, and accepted again on the earlier assistant turns of the
    next request, so a caller can replay them

If there is already a way to do this in Data Designer, a pointer would be just as good, which is why
this is filed as a question as much as a request.

Describe alternatives you've considered
  • Chat Completions only: works, but the reasoning is lost between turns (the problem above).
  • litellm's Responses bridge outside Data Designer: what MindEval does now. It works, but those calls
    bypass Data Designer's model layer, so User Sim's accounting, retries and model configuration don't
    apply to them.
  • A proxy in front of Data Designer (e.g. a LiteLLM proxy translating /chat/completions to
    /responses): gets the call through, but the reasoning items come back as an extra field that Data
    Designer's parser drops, and there is no way to send them back on the next turn.
  • previous_response_id with store: true: keeps the reasoning on OpenAI's side instead of
    carrying it. Not usable where data must not be stored, and it ties each turn to one provider session.
Agent Investigation

Checked on main (2026-10-14) and in 0.9.4:

  • models/clients/factory.py: _SUPPORTED_PROVIDER_TYPES = ("openai", "anthropic"); any other
    provider type raises. Plugins can't add a model client (plugin types: column generator, seed reader,
    processor).
  • models/clients/adapters/openai_compatible.py: chat calls always post to _ROUTE_CHAT = "/chat/completions"; no other route for chat.
  • models/clients/parsing.py: parse_assistant_message keeps content, reasoning_content (text),
    tool_calls and images; other message fields, such as reasoning items, are dropped.
  • No existing issue found ("responses api", "encrypted_content", "/v1/responses").
  • In MindEval (mindeval/llm.py), litellm returns the items as reasoning_items (or under
    provider_specific_fields) on the message; the benchmark stores them with the turn and sends them back
    on that turn next time (mindeval/session.py, CARRIED = ("reasoning_content", "thinking_blocks", "reasoning_items")).
Second gap: Vertex AI

The same benchmark uses Claude and Gemini on Vertex AI, today through litellm's vertex_ai/ provider,
outside Data Designer's model layer. What blocks it, as far as we can see in the code (main):

  • Claude on Vertex (per Google Cloud's API docs) speaks the Anthropic Messages format, but at
    …/publishers/anthropic/models/{model}:rawPredict, with the model in the URL,
    anthropic_version: "vertex-2023-10-16" in the body, and a Google access token. The Anthropic adapter
    (models/clients/adapters/anthropic.py) posts to {endpoint}/v1/messages with x-api-key and the
    model in the body.
  • Gemini on Vertex works through Vertex's OpenAI-compatible endpoint, but authenticates with a Google
    access token that expires after about an hour, while a provider's key is resolved once when the client
    is built (models/clients/factory.py, _resolve_api_key). A long run fails once the token expires.

Useful pieces: Google credentials (Application Default Credentials or a service account) with token
refresh during a run, and the Vertex URL and body shape for Claude.

Additional context

Related: User Sim's opt-in replay of the assistant's own reasoning (NVIDIA-NeMo/UserSim#31) currently
covers OpenAI-compatible endpoints that return reasoning_content; this would extend it to OpenAI's own
reasoning models.

  • I've reviewed existing issues and the documentation
  • This is a design proposal, not a "please build this" request
Lenguaje dominante
Python
Estrellas
2.3k
Forks
227
Merge medio
22 h 51 min
PR fusionados (30 d)
61

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA-NeMo/DataDesigner

Todos los issues de NVIDIA-NeMo/DataDesigner

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.