ggml-org/llama.cpp

llama : refactor the llm.build_xxx functions

已關閉

#5,239 建立於 2024年1月31日

 (3 則留言) (3 個反應) (0 位負責人)C++ (21,861 個分叉)batch import
good first issuerefactoringroadmap

倉庫指標

星標
 (124,507 顆星)
PR 合併指標
 (平均合併 6天 8小時) (30 天內合併 389 個 PR)

描述

Now that we support a large amount of architectures, we can clearly see the patterns when constructing the compute graphs - i.e. optional biases, different norm types, QKV vs Q+K+V, etc.

We should deduplicate the copy-paste portions in functions such as llm.build_llama(), llm.build_falcon(), etc.

The advantage of the current code is that it is easy to look into the graph of a specific architecture. When we refactor this, we will lose this convenience to some extend. So we should think about making this refactoring in such a way that we don't completely obscure which parts of the graph belong to which architectures

Open for ideas and suggestions how to do this best

貢獻者指南