lm-sys/FastChat

Support for htttps protocol call between controller node and worker node

开放

#1,915 创建于 2023年7月10日

 (4 条评论) (1 个反应) (0 位负责人)Python (4,736 个派生)batch import
good first issue

仓库指标

星标
 (38,959 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Hi team,

Thanks for the great work. Recently, we are trying to set up a large-scale chatbot service. Our case is like: We have a controller server node where we can set up a web application; we have another server which contains 8 gpus to run the model instances. However, the GPU server we have can only support the https calls rather than http calls because only the default 443 https port can be opened to listen. We tried to enable the controller to call the https 443 port of GPU server node and the following error was received on controller node server:

Get status fails: https://GPU_Server_ip:443, HTTPSConnectionPool(host='GPU_Server_ip', port=443): Max retries exceeded with url: /worker_get_status (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)')))

Is there anyway we can use htttps call to get the message from the server?

贡献者指南