llama : add CLI assistant · ggml-org/llama.cpp#10688

(4 comments) (3 reactions) (0 assignees)C++ (110,169 stars) (18,202 forks)batch import

enhancementgood first issueroadmap

説明

The https://github.com/AnswerDotAI/shell_sage project seems quite fun and useful. GitHub Copilot CLI was also a rather useful utility.

We should build a fully-local alternative leveraging llama-server's advanced context reuse techniques in the same spirit as llama.vim. Speculative decoding should be very effective in this scenario as well. Likely the Qwen 2.5 family would be ideal for this use case.

コントリビューターガイド

技術スタック: cppshell
領域: clibackend
Issue 種別: feature
難度: 4
推定時間: over 1 week
活動状況: fresh
明確さ: needs investigation
前提条件: C++ knowledgeshell scriptingfamiliarity with llama.cpp
初心者向け度: 20
調査方針: The issue proposes building a CLI assistant similar to shell sage and GitHub Copilot CLI, leveraging llama server's context reuse and speculative decoding. To start, examine the existing llama.vim implementation and the shell sage project for design patterns. Investigate llama server's API for context reuse and how to integrate speculative decoding. The Qwen 2.5 family is suggested as a model choice. The exact CLI interface and command structure need to be defined, possibly through a community discussion.