Add withoutBG open ONNX alpha-matting / background-removal model to Inference

Aperta
  • Qualcosa non va?
#2,764 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
huggingface, numpy, python
Ambito
backend-api-design, computer-vision, machine-learning

Direzione di ricerca

Inizia leggendo il workflow depth_estimation@v1 esistente e i pattern dei modelli in inference_models. Controlla come sono organizzati i registri dei modelli, i test delle predizioni e i test di letterboxing e unpadding. Il lavoro è completato quando il modello ONNX withoutBG dispone di un task e di un confezionamento dei pesi concordati, della copertura dell’integrazione, della documentazione e degli avvisi richiesti.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement
Search before asking
  • I have searched the Inference issues and found no similar feature requests.
Description

I'd like to add withoutBG Open Weights as a foundation alpha matting / background removal model to inference / inference_models.

Model: https://huggingface.co/withoutbg/withoutbg-openweights-onnx
I'm the author and trainer of this model.

What it is
  • Self-contained ONNX model (opset 18, ~455 MB, fp32)
  • Pipeline combines Depth Anything V2 (embedded in the graph) with a ConvNeXt-based matting head built on a DINOv3 ConvNeXt backbone
  • No trimap required. Takes an RGB image as input and produces a soft alpha matte
  • Fixed 448×448 input using letterboxing. A sidecar JSON documents the input/output names, tensor shapes, and SHA-256 checksum
I/O contract
Name Shape Dtype Range
Input rgb [1, 3, 448, 448] float32 [0, 1] NCHW
Output alpha [1, 1, 448, 448] float32 [0, 1]

Pre/post-processing is straightforward:

  1. Letterbox the longest side to 448.
  2. Pad to the top left.
  3. Run the ONNX model.
  4. Remove the padding.
  5. Resize the alpha matte back to the original image size.
  6. Optionally produce an RGBA cutout.
Proposed integration (v1)
  1. Add an ONNX model class to inference_models with from_pretrained, plus a registry entry under a new task such as image-matting or background-removal (happy to follow the maintainers' preference).
  2. Add a workflow block modeled after depth_estimation@v1 that outputs alpha (NUMPY_ARRAY_KIND) together with an optional cutout or visualization image.
  3. Add documentation, unit tests for letterboxing and unpadding, a prediction test, and the required license and third-party notices.
  4. Add an HTTP /infer adapter later if it makes sense.
Licensing
  • Model weights: Apache-2.0
  • Third-party components:
    • Depth Anything V2: Apache-2.0
    • DINOv3: Meta's DINOv3 license, consistent with how DINOv3 classification models are already documented in this repository
Questions
  1. What task type would you prefer for the registry?
  2. How would you prefer to package the weights and assign model IDs? Roboflow model package or direct download from Hugging Face?
  3. Any concerns about shipping a fused ONNX model that includes DINOv3 ConvNeXt components?

I'm happy to put together the implementation once we're aligned on the above.

Use case
  • Background removal (soft alpha, not a hard mask)
  • Product & portrait isolation where fine details matter (hair, edges) and instance / semantic segmentation is not enough
  • Image compositing (alpha + original -> cutout / new background)
Additional

See model outputs: https://withoutbg.com/open-model/results

Image
Are you willing to submit a PR?
  • Yes I'd like to help by submitting a PR!
Lingua principale
Python
Stelle
2.5k
Fork
340
Merge medio
2g 19h
PR unite (30g)
92

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di roboflow/inference

Tutte le issue di roboflow/inference

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.