Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Lend part of a card from a running engine and get it back, without a park or a reload

オープン
  • 何か問題がありますか？
#189 コメント 26 件 リアクション 1 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@T-Crypt がすでに取り組んでいます。

2026年10月23日 から。

評価

この issue はまだ評価されていません。

説明

Project home: https://github.com/TacoTakumi/vram-lend

My setup

  • Ubuntu 24.04, NVIDIA driver 610.43.02, CUDA 13.
  • 2x RTX 3090 (24 GB), 1x RTX 5060 Ti (16 GB) and 2x RTX 4060 Ti (16 GB). 96 GB of VRAM, 128 GB RAM.
  • llama-swap serves everything. Most models run on llama.cpp. GLM-5.3-Flash and DeepSeek V4.1 Flash run on Project Maya, Qwen 3.8 Flash Next runs on Strata.
  • Right now Maya runs GLM-5.3-Flash and DSV41 as three llama-swap entries, on 2, 3 or 5 cards.
  • ComfyUI runs under llama-swap on the 5060 Ti.
  • VoiceManager runs on a 4060 Ti.

What I want to do

When I want ComfyUI or a voice model, I switch Maya from the 5 card entry to the 2 or 3 card entry. The model reloads and I lose the session cache. When I am done I switch back and it reloads again.

Maya keeps the experts in system RAM and a cache of them on the cards. It can already give part of that cache away inside its own process. The vision encoder starts only when a picture arrives, in VRAM that the model lends from its expert cache. I want the same thing between processes. Maya drops part of its cache on one card and keeps serving, only slower. ComfyUI or the voice model uses that memory. When the borrower is done, Maya takes the memory back and fills its cache again.

So the engine does not stop. It lends N MiB on one named card and gets it back later. Park is all or nothing and needs-mib is fixed at the grant, so neither covers this. It is close to the fit and scenes item on your roadmap, but here the holder keeps running.

Questions

  1. Would you take a lease that its holder can shrink and grow while it serves? The flow I have in mind:
  • A borrower asks for N MiB on card X.
  • Berth asks the holder to give back N MiB on card X.
  • The holder answers when the memory is free. Berth checks the driver read, as it does for a release now.
  • Berth grants the borrower a sized lease.
  • When the borrower ends, Berth tells the holder it can take the memory back.
  1. Should this go in the park protocol? Something like this would work for Maya and Strata:
POST /berth/lend    {"card": "0000:05:00.0", "mib": 8192}  -> 200 when the memory is free, 409 if the engine cannot give that much
POST /berth/return  {"card": "0000:05:00.0"}               -> 200 when the engine holds it again
GET  /berth/state   -> {"state": "hot", "lent": {"0000:05:00.0": 8192}}

  1. My model holds five cards and lends from one of them. Will the v0.4 lease table let one lease cover several cards, with a size for each card?

  2. I will build this in my own forks of Berth, Maya and Strata. If the shape fits where Berth is going, would you take the Berth side back as a PR?

Thanks!

主要言語
Go
スター
226
フォーク
20
平均マージ
51分
マージ済み PR（30日）
225

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

T-Crypt/berth のほかの issue

T-Crypt/berth の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。