macOS on Apple silicon: `--gpus all` as a private channel to a GPU server on the Mac
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- macos, rust
- Ambito
- infrastructure, networking
Direzione di ricerca
Read the issue’s linked patch against commit 260cf66 and the repository’s CONTRIBUTING guidance on cross-platform features. Start by tracing how GPU device requests are handled and how the guest agent and daemon communicate; the issue identifies nine changed files and added unit tests. Done means the maintainers agree on the macOS-only relay approach and its platform behavior, then the change meets the project’s support and validation requirements.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
What you're trying to do
I run MLX and PyTorch programs in containers on an Apple-silicon Mac. In a container they compute on the CPU, because Metal does not reach into the Linux virtual machine. I wrote two bridges that send a program's array operations from the container to a process on the Mac, which computes them on the Apple GPU; the programs are not changed (https://github.com/FabioMalpezzi/varco).
With Docker Desktop the container reaches that process on a TCP port of the Mac, and so does every other container. With Kiri I wanted the container started with --gpus all, and only that one, to have the channel.
What you'd like Kiri to do
Accept a GPU device request on an Apple-silicon Mac, where Kiri refuses it today, and give that container a private channel to a Unix socket on the Mac. Kiri carries bytes and does nothing with the GPU itself.
I have this working as a patch against 260cf66 (main on 16 November): https://github.com/FabioMalpezzi/varco/tree/main/kiri
- A container created with
--gpus all(or Compose'scapabilities: [gpu]) gets a listener at127.0.0.1:47700in its own network namespace, opened by the guest agent before the program starts, and the variableKIRI_GPU_BRIDGE. - Each connection is relayed over vsock to the daemon, which connects it to
~/.kiri/run/gpu.sock, where a server of the user's listens. With no server there the container sees its connection close. --gpustogether with--network hostor--network container:...is refused, and no container may join the network namespace of a--gpuscontainer, at create and at start.- On every other host a GPU request is refused as it is now.
It is nine files, with unit tests for the request and for the refusals. Through it, on an M4 Max, BGE-M3 embeddings took 1.01 to 1.02 times the Mac's time (1.11 through Docker Desktop's TCP port), and a 35B model generated the Mac's text at the Mac's speed. The figures and the method are in https://github.com/FabioMalpezzi/varco/blob/main/docs/benchmarks.md
What I know does not fit your rules as they stand, and would like your view on before any pull request:
- It is for one system. On Windows and Linux hosts there is no Metal to reach, so the request stays refused there, and CONTRIBUTING asks that a feature ship on all three.
- What Kiri would gain is a relay: a container asks for a device and gets a socket on the host. The names (
KIRI_GPU_BRIDGE, port 47700,gpu.sock) are mine and can change. - I ran the unit tests the patch adds or changes, and the channel on a daemon built from source at
260cf66. I did not runtests/e2e.sh, the Speed check, or builds for Linux, Intel Macs and Windows.
I am not a native English speaker, so I used an AI tool to write this message. The patch was also written with the help of an AI coding tool; I built it, ran it and measured with it on my own Mac.
Would you like to build it?
Yes, once we've agreed on the approach
- Lingua principale
- Rust
- Stelle
- 7.5k
- Fork
- 329
- Merge medio
- 7h 48m
- PR unite (30g)
- 105
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di libracoder/kiri
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
libracoder/kiri#364 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
libracoder/kiri#289 ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 47/100
libracoder/kiri#365 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 35/100
libracoder/kiri#362 ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 3/5 1-2 giorni Idoneità per principianti 47/100
libracoder/kiri#359 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di libracoder/kiri
Issue simili
-
benchmark-review
Difficoltà 2/5 1-3 ore Idoneità per principianti 63/100
I maintainer di solito rispondono entro 1 giorno
-
area:filesystem bug need-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
Captions panel does not offer to show an existing transcriptForse già presa @thedavidweng l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
CapSoftware/Cap#4276 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
matrix-org/matrix-rust-sdk#7495 ·
I maintainer di solito rispondono entro 1 giorno