Misc. bug: server provides strutured output for response_format: json_object, but not for response_format: json_schema
#10,732 建立於 2024年12月9日
倉庫指標
- 星標
- (124,134 顆星)
- PR 合併指標
- (平均合併 6天 8小時) (30 天內合併 389 個 PR)
描述
Name and Version
on latest commit ce8784bdb153ff7794dde5a50b0ebfa51baa6171
but have been noticing it for several days now
Operating systems
No response
Which llama.cpp modules do you know to be affected?
No response
Problem description & steps to reproduce
I have been trying to follow the steps for structured json output using json_schema in server here
however, I was not able to get any combination of json_schema to work. I was able to get json_object to work, doing effectively the same thing, but since this differs from the OpenAI API (which I suppose server is striving for) I suppose it's a bug. The official server docs also mention json_schema is supported (see the above link)
Does not work, does not apply any structured json schema at all, and does not indicate any failure or warning (simply returns unstructured output), taken directly from OpenAI's docs (click "curl" for the curl version)
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "math_reasoning",
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": {
"type": "string"
},
"output": {
"type": "string"
}
},
"required": [
"explanation",
"output"
],
"additionalProperties": false
}
},
"final_answer": {
"type": "string"
}
},
"required": [
"steps",
"final_answer"
],
"additionalProperties": false
},
"strict": true
}
}
however, "json_object" works as expected
"response_format": {
"type": "json_object",
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": {
"type": "string"
},
"output": {
"type": "string"
}
},
"required": [
"explanation",
"output"
],
"additionalProperties": false
}
},
"final_answer": {
"type": "string"
}
},
"required": [
"steps",
"final_answer"
],
"additionalProperties": false
}
}
First Bad Commit
No response
Relevant log output
No response