Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Support Qwen3.8-Flash-Next-GSQ-RCO GGUF and integrate Strata + KVMem

オープン
  • 何か問題がありますか？
#658 コメント 0 件 リアクション 2 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
10/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
活発
技術スタック
python
領域
machine-learning

調査の方向性

The payload names no files, tests or entry points. Start by finding the existing model loading and KV cache or memory management code in the repository, and check the linked Roadmap (#79) and FAQ (#84) for any existing plan. The three requests (GGUF support, Strata, KVMem) are separate large integrations, so done would mean a narrowed sub-issue with a stated scope and acceptance test, not a patch to this thread.

索引モデルが issue の本文から書いたものです。

説明

Desktop feature
Before you start
  • I have read the Roadmap and this is not already planned there.
  • I have read the FAQ.
  • I have searched existing issues and found no duplicate.
Applies to

Desktop app

What problem does this solve

Hi FreeToken team,

I would like to request support for Qwen3.8-Flash-Next-GSQ-RCO GGUF. Many users are waiting to run this model locally, and FreeToken's edge-native MoE design looks like a great fit. Native GGUF support would make it much easier to load, test, and use this model without extra conversion work.

I also hope you can consider merging/integrating the Strata inference framework. Strata is built for MoE models that cannot fully fit in VRAM, splitting weights across GPU VRAM, host RAM, and NVMe while keeping at least 4-bit precision. This fits FreeToken's goal of efficient local inference and could help run large models like Qwen3.8-Flash-Next on consumer hardware.

Please also consider integrating KVMem. Long-context agent workflows can quickly run out of VRAM, while KVMem keeps old KV history in host RAM and only loads relevant blocks into a bounded GPU working set. That would greatly improve long conversations, tool calls, and long-running sessions in FreeToken.

We really need these features. If any of them are already planned, please let us know. I am happy to help test or provide more details. Thank you!

Proposed solution

Add native support for Qwen3.8-Flash-Next-GSQ-RCO GGUF, so users can load and run this model directly in FreeToken without extra conversion work.

Integrate the Strata inference framework into FreeToken. Strata is designed for MoE models that cannot fully fit in VRAM. It splits weights across GPU VRAM, host RAM, and NVMe while keeping at least 4-bit precision. This fits FreeToken's goal of efficient local inference and would help run large models like Qwen3.8-Flash-Next on consumer hardware.

Integrate KVMem into FreeToken as an optional KV cache / context manager. KVMem keeps long KV history in host RAM and only loads relevant blocks into a bounded GPU working set. This would improve long-context chat, tool calls, and long-running agent sessions without using too much VRAM.

Ideally, expose these features as config options so users can enable or disable them. Please also keep compatibility with FreeToken's current caching and memory management. I am happy to help test if needed.

Alternatives considered

No response

主要言語
Python
スター
14.2k
フォーク
1.4k
平均マージ
21時間 17分
マージ済み PR（30日）
97

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

FlashML-org/FreeToken のほかの issue

FlashML-org/FreeToken の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。