Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Feature: Support LeRobot Pi 0.5 policies with native Model Connect bundles

Chiusa
  • Qualcosa non va?
#1,880 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@chaofengw-nv ci sta già lavorando.

Dal 29/10/2026.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
huggingface, python, pytorch
Ambito
backend, machine-learning

Direzione di ricerca

Start by reviewing the prior Pi 0.5 implementation findings in closed PR #534 and the repository’s current model-family, builder, native Task, and E2E test patterns. Pin and verify access to the stated LeRobot checkpoint revision before beginning the model proof; compare the complete action chunk against the LeRobot reference with matched inputs and starting noise. Done means the checkpoint builds into a native bundle, the declared LeRobot contract works, numerical checks pass, and the qualified runtime details and limits are documented.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

AI enhancement model-onboarding

This issue was drafted by AI at a user's request.

Request type

New model or checkpoint support

Problem and use case

A LeRobot user should be able to point lerobot-export at a Pi policy produced by lerobot-train, select TensorRT-Model-Connect as the compiler/runtime backend, and receive a native bundle that LeRobot can load for action inference. Model Connect currently has a qualified LeRobot ACT family, but no Pi family. Its existing image-and-state action Task cannot represent Pi's named cameras and language task.

Proposed outcome

Add family-owned support for LeRobot's standard Pi 0.5 (pi05) flow-matching policy. The builder should consume a supported LeRobot checkpoint directory, including its config, weights, saved processors, tokenizer, and normalization state, without a manual conversion step. The native runtime should accept the supported observation and task contract and return complete actions in robot units. Expose a versioned, model-agnostic Task interface where the current one-image contract is insufficient, so LeRobot can call the bundle through its export/rollout integration.

The first slice is Pi 0.5 without visual or proprioceptive memory. Pi 0, Pi 0-FAST, Pi 0.7, and optional memory or real-time chunking modes require separate qualification before being claimed as supported.

Acceptance criteria
  • Pin an accessible public LeRobot Pi 0.5 checkpoint and exact revision for the first proof; verify CI can fetch every required asset before the model proof starts.
  • trtmc build or the public builder API produces a bundle from the LeRobot checkpoint and from a compatible lerobot-train output, without checkpoint-name-only behavior or manual weight rewriting.
  • A native Model Connect Task accepts the declared camera, state, and task inputs and returns the full unnormalized action chunk; unsupported shapes, missing required inputs, and configurations fail clearly.
  • Family-owned E2E tests compare all action values against the pinned LeRobot pi05 reference on representative recorded observations with matched starting noise and processing, using meaningful numerical criteria.
  • Validate one documented precision and NVIDIA GPU/TensorRT stack, and report build, startup, memory, and action-chunk latency. Native inference has no Python or PyTorch dependency.
  • Document the supported checkpoint/configuration boundary and the bundle/runtime contract needed by lerobot-export and lerobot-rollout.
Model and target details
  • Initial public checkpoint: lerobot/pi05_libero_base, revision 5fcf06fc42b6717e22ab885fd381a096574b12c7 (public model metadata and checkpoint files were available when this issue was filed).
  • LeRobot policy type: pi05; initial task: vision/language/state to robot action chunk, batch size one.
  • Initial precision candidate: BF16; pin the exact qualified precision, GPU, CUDA, and TensorRT versions in the implementation evidence. Jetson deployment requires its own target qualification.
  • Expected path: LeRobot-trained checkpoint -> Model Connect builder -> .bundle -> native Task API -> LeRobot export/rollout adapter.
Alternatives considered

Shoumikhin's LeRobot export prototype offers ONNX-TensorRT, ExecuTorch-TensorRT, and Torch-TensorRT paths. Those provide useful reference behavior but do not build or execute Model Connect bundles.

Additional context

Yifei's closed OpenPI Pi 0.5 PR #534 implemented and locally validated a native DROID policy path. It did not merge: its required prepared checkpoint was unavailable to CI, and the isolated model proof failed at asset readiness. Reuse its family-owned implementation findings where applicable, while proving the new LeRobot checkpoint contract with accessible immutable assets and current repository architecture. Do not weaken validation to bypass asset or numerical failures.

Submission checks
  • I searched open and closed issues and found no duplicate request.
  • I removed secrets, private/internal evidence, personal paths, and restricted artifacts.
Lingua principale
Python
Stelle
283
Fork
75
Merge medio
1g 23h
PR unite (30g)
227

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/TensorRT-Model-Connect

Tutte le issue di NVIDIA/TensorRT-Model-Connect

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.