macOS on Apple silicon: `--gpus all` as a private channel to a GPU server on the Mac

オープン
#356 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
macos, rust

調査の方向性

Read the issue’s linked patch against commit 260cf66 and the repository’s CONTRIBUTING guidance on cross-platform features. Start by tracing how GPU device requests are handled and how the guest agent and daemon communicate; the issue identifies nine changed files and added unit tests. Done means the maintainers agree on the macOS-only relay approach and its platform behavior, then the change meets the project’s support and validation requirements.

索引モデルが issue の本文から書いたものです。

説明

What you're trying to do

I run MLX and PyTorch programs in containers on an Apple-silicon Mac. In a container they compute on the CPU, because Metal does not reach into the Linux virtual machine. I wrote two bridges that send a program's array operations from the container to a process on the Mac, which computes them on the Apple GPU; the programs are not changed (https://github.com/FabioMalpezzi/varco).

With Docker Desktop the container reaches that process on a TCP port of the Mac, and so does every other container. With Kiri I wanted the container started with --gpus all, and only that one, to have the channel.

What you'd like Kiri to do

Accept a GPU device request on an Apple-silicon Mac, where Kiri refuses it today, and give that container a private channel to a Unix socket on the Mac. Kiri carries bytes and does nothing with the GPU itself.

I have this working as a patch against 260cf66 (main on 16 November): https://github.com/FabioMalpezzi/varco/tree/main/kiri

  • A container created with --gpus all (or Compose's capabilities: [gpu]) gets a listener at 127.0.0.1:47700 in its own network namespace, opened by the guest agent before the program starts, and the variable KIRI_GPU_BRIDGE.
  • Each connection is relayed over vsock to the daemon, which connects it to ~/.kiri/run/gpu.sock, where a server of the user's listens. With no server there the container sees its connection close.
  • --gpus together with --network host or --network container:... is refused, and no container may join the network namespace of a --gpus container, at create and at start.
  • On every other host a GPU request is refused as it is now.

It is nine files, with unit tests for the request and for the refusals. Through it, on an M4 Max, BGE-M3 embeddings took 1.01 to 1.02 times the Mac's time (1.11 through Docker Desktop's TCP port), and a 35B model generated the Mac's text at the Mac's speed. The figures and the method are in https://github.com/FabioMalpezzi/varco/blob/main/docs/benchmarks.md

What I know does not fit your rules as they stand, and would like your view on before any pull request:

  • It is for one system. On Windows and Linux hosts there is no Metal to reach, so the request stays refused there, and CONTRIBUTING asks that a feature ship on all three.
  • What Kiri would gain is a relay: a container asks for a device and gets a socket on the host. The names (KIRI_GPU_BRIDGE, port 47700, gpu.sock) are mine and can change.
  • I ran the unit tests the patch adds or changes, and the channel on a daemon built from source at 260cf66. I did not run tests/e2e.sh, the Speed check, or builds for Linux, Intel Macs and Windows.

I am not a native English speaker, so I used an AI tool to write this message. The patch was also written with the help of an AI coding tool; I built it, ran it and measured with it on my own Mac.

Would you like to build it?

Yes, once we've agreed on the approach

主要言語
Rust
スター
7.5k
フォーク
329
平均マージ
7時間 48分
マージ済み PR(30日)
105

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

libracoder/kiri のほかの issue

libracoder/kiri の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。