Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Speed up inference performance for attention and ssm layers with caching

未关闭
  • 哪里不对？
#279 0 条评论 1 个 reaction 已指派 1 人 在 GitHub 查看

@bigximik 已经在做这个了。

开始于 2025年6月30日。

评估

这个 Issue 还没有评估数据。

描述

enhancement need update

🎯 Goal (What & Why)

Speed up inference performance for attention layers with KV caching

Implemented based on functionality in Hugging Face Transformers. Needs to support data, model, and sequence parallelism.

🚀 Execution Plan

Step 1

support range of positions starting from > 0 in fast-llm inference which is needed for HF generate with cache as they send only the last token to model forward

Step 2: W

implement KV cache for HF generte

📌 Acceptance Criteria (Must-Haves for Completion)

  • The feature must be functional and tested.
  • The implementation must be documented in practical terms.
  • The PR must include a performance/impact summary.
  • No refactors unless directly necessary for feature completion.

🛠️ Project Management

  • Assign the project to the Fast-LLM project.
  • Set the Estimate field (in days) in the GitHub project.
  • Use the Size field to categorize the PR size (Small/Medium/Large).
  • Assign an owner when opening the issue.
主要语言
Python
星标
337
派生
52
30 天内没有已合并 PR

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

ServiceNow/Fast-LLM 的其他 Issue

查看 ServiceNow/Fast-LLM 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。