对LLM API的Stream请求会自动添加inlude_usage选项，需要增加一个配置开关 || Stream requests to the LLM API will automatically add the include_usage option, and a configuration switch needs to be added.

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

• I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

对LLM API的Stream请求会自动添加inlude_usage选项，使不支持的LLM API路由失败。

Ⅱ. Describe what happened

按正常流程注册AI服务和路由，stream: False时请求正常响应，stream: True时请求报错，具体报错内容转发自AI服务端。跳过Higress网关请求LLM API并带stream: True则正常响应。 排查后发现时Higress为统计token用量在转发时添加inlude_usage: True，该选项虽符合OpenAI标准，但对于一些旧版本推理引擎可能存在不兼容问题。

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

建议提供逃生通道或在响应中显式提醒。 （可选）在LLM服务注册时添加一键测试常见功能的API兼容性。 （可选）网关侧部署本地tokenizer辅助token统计。

Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. 部署旧版本vllm模型镜像（问题LLM API的vllm版本为0.4.3）
2. 注册LLM API服务至网关并配置路由
3. 通过网关请求该模型，并添加stream: True参数

结果：响应400 信息：{"object":"error","message":"[{'type': 'extra_forbidden', 'loc': ('body', 'stream_options'), 'msg': 'Extra inputs are not permitted', 'input': {'include_usage': True}}]","type":"BadRequestError","param":null,"code":400}

Ⅴ. Anything else we need to know?

It is recommended to provided Higress runtime logs and configurations for us to investigate your issue, especially for controller and gateway components.

Please checkout following documents on how to obtain these data.

• https://higress.cn/docs/latest/ops/how-tos/view-logs/
• https://higress.cn/docs/latest/ops/how-tos/view-configs/

Ⅵ. Environment:

• Higress version:
• OS:
• Others:

---

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

• I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

Stream requests to the LLM API automatically add the include_usage option, causing unsupported LLM API routes to fail.

Ⅱ. Describe what happened

Register the AI service and route according to the normal process. When stream: False, request a normal response. When stream: True, request an error. The specific error content is forwarded from the AI server. Skip the Higress gateway and request the LLM API with stream: True to respond normally. After investigation, it was found that Higress added include_usage: True when forwarding to count token usage. Although this option complies with the OpenAI standard, it may have incompatibility issues with some older versions of inference engines.

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

It is recommended to provide an escape route or include an explicit reminder in the response. (Optional) Add one-click testing of API compatibility of common functions when registering the LLM service. (Optional) Deploy a local tokenizer on the gateway side to assist in token statistics.

Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. Deploy the old version of vllm model image (the vllm version of the LLM API in question is 0.4.3)
2. Register the LLM API service to the gateway and configure routing
3. Request the model through the gateway and add the stream: True parameter

Result: Response 400 Message: {"object":"error","message":"[{'type': 'extra_forbidden', 'loc': ('body', 'stream_options'), 'msg': 'Extra inputs are not permitted', 'input': {'include_usage': True}}]","type":"BadRequestError","param":null,"code":400}

Ⅴ. Anything else we need to know?

It is recommended to provided Higress runtime logs and configurations for us to investigate your issue, especially for controller and gateway components.

Please checkout following documents on how to obtain these data.

• https://higress.cn/docs/latest/ops/how-tos/view-logs/
• https://higress.cn/docs/latest/ops/how-tos/view-configs/

Ⅵ. Environment:

-Higress version: -OS:

• Others: