ggml-org/llama.cpp

Misc. bug: server provides strutured output for response_format: json_object, but not for response_format: json_schema

Aperta

#10.732 aperta il 9 dic 2024

 (6 commenti) (3 reazioni) (0 assegnatari)C++ (21.737 fork)batch import
enhancementgood first issue

Metriche repository

Star
 (124.043 stelle)
Metriche merge PR
 (Merge medio 6g 8h) (389 PR mergiate in 30 g)

Descrizione

Name and Version

on latest commit ce8784bdb153ff7794dde5a50b0ebfa51baa6171

but have been noticing it for several days now

Operating systems

No response

Which llama.cpp modules do you know to be affected?

No response

Problem description & steps to reproduce

I have been trying to follow the steps for structured json output using json_schema in server here

however, I was not able to get any combination of json_schema to work. I was able to get json_object to work, doing effectively the same thing, but since this differs from the OpenAI API (which I suppose server is striving for) I suppose it's a bug. The official server docs also mention json_schema is supported (see the above link)

Does not work, does not apply any structured json schema at all, and does not indicate any failure or warning (simply returns unstructured output), taken directly from OpenAI's docs (click "curl" for the curl version)

    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "math_reasoning",
        "schema": {
          "type": "object",
          "properties": {
            "steps": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "explanation": {
                    "type": "string"
                  },
                  "output": {
                    "type": "string"
                  }
                },
                "required": [
                  "explanation",
                  "output"
                ],
                "additionalProperties": false
              }
            },
            "final_answer": {
              "type": "string"
            }
          },
          "required": [
            "steps",
            "final_answer"
          ],
          "additionalProperties": false
        },
        "strict": true
      }
    }

however, "json_object" works as expected

 "response_format": {
    "type": "json_object",
    "schema": {
        "type": "object",
        "properties": {
            "steps": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "explanation": {
                            "type": "string"
                        },
                        "output": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "explanation",
                        "output"
                    ],
                    "additionalProperties": false
                }
            },
            "final_answer": {
                "type": "string"
            }
        },
        "required": [
            "steps",
            "final_answer"
        ],
        "additionalProperties": false
    }
}

First Bad Commit

No response

Relevant log output

No response

Guida contributor