TabbyML/tabby

Support serve model DeepSeek R1 and disable thinking process in Answer ouput

Open

#3,977 建立於 2025年3月10日

在 GitHub 查看
 (2 留言) (5 反應) (0 負責人)Rust (1,744 fork)batch import
enhancementgood first issue

倉庫指標

Star
 (33,513 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

How to load a reasoning model DeepSeek R1 in Tabby?

I connected to Ollama via config.toml.

[model.chat.http]
kind = "openai/chat"
model_name = "DeepSeek-R1-Distill-Qwen-7B"
api_endpoint = "XXXXXXXX"
chat_template = "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are \"Tabby\", a conscious sentient superintelligent artificial intelligence designed for helping software developers. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}<|im_start|>assistant\n"

I got the tag <think>XXXXXX</think> in the answer. I want to disable thinking process in the answer output. Thank you!

貢獻者指南