Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

HTTP models ignore `includeSchemaInPrompt`, so structured output fails on servers that don't enforce schemas

Aberta
  • Algo errado?
#369 2 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
25/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
swift
Domínio
backend-api-design

Direção de pesquisa

Start by tracing how includeSchemaInPrompt is handled in the existing MLX, llama.cpp, and Core ML models, then compare that with the Ollama, OpenAI, Open Responses, Anthropic, and Gemini request paths. The issue describes adding schema instructions to the last transcript entry before provider message conversion, while retaining each server's schema field. Done means streaming, non-streaming, and post-tool-round tests cover every HTTP model; a proposed fix and tests are already available in a linked branch.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

The HTTP language models send a response schema only to the server (format, response_format, output_config, responseSchema), and ignore includeSchemaInPrompt. When the server doesn't enforce the schema, the model never sees it, answers in prose, and respond(generating:) throws typeMismatch.

Ollama's cloud models are one such server: they accept format and ignore it (ollama/ollama#12362). Some OpenAI-compatible servers do the same with response_format.

To reproduce
@Generable
struct Greeting {
    let text: String
}

let session = LanguageModelSession(model: OllamaLanguageModel(model: "deepseek-v4.1-flash:cloud"))
let greeting = try await session.respond(to: "Say hello.", generating: Greeting.self)
// Throws DecodingError.typeMismatch: the model answered in plain text.

The same code works with a local model such as qwen3:8b, because the local Ollama server enforces format.

Cause

includeSchemaInPrompt is true by default. MLX, llama.cpp, Core ML and Foundation Models honor it by saying the schema in the prompt. Ollama, OpenAI, Open Responses, Anthropic and Gemini accept it but never use it.

Proposed fix

Make the HTTP models honor includeSchemaInPrompt the way the local ones do:

  • When includeSchemaInPrompt is true and the response type isn't String, each HTTP model (Ollama, OpenAI, Open Responses, Anthropic, Gemini) appends the schema instructions as a text segment to the last prompt entry of the request's transcript, before converting it to the provider's messages.
  • The schema is still sent to the server as today (format, response_format, output_config, responseSchema).
  • The private schemaPrompt(for:) copies in MLX, llama.cpp and Core ML become one shared GenerationSchema.schemaPrompt(), which states the inlined schema.

A branch with this and tests for every HTTP model (streaming, non-streaming, after a tool round) is ready as a PR: akbashev:fix/schema-in-prompt.

Still a problem after the fix

Servers that enforce a schema constrain decoding, so the model can only produce valid output. Servers that don't, like Ollama Cloud (ollama/ollama#12362), let it write anything: the schema in the prompt is only an instruction, and models sometimes ignore it, especially after tool calls. With beam:cloud, about half of the answers after several tool calls were still prose, so typeMismatch.

Linguagem predominante
Swift
Estrelas
966
Forks
103
Merge médio
1d 3h
PRs com merge (30d)
64

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de huggingface/AnyLanguageModel

Todas as issues de huggingface/AnyLanguageModel

Issues semelhantes

Mais issues de Swift

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.