Add withoutBG open ONNX alpha-matting / background-removal model to Inference

Open
  • Is something wrong?
#2,764 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
huggingface, numpy, python
Domain
backend-api-design, computer-vision, machine-learning

Research direction

Start by reading the existing depth_estimation@v1 workflow and the model patterns in inference_models. Check how model registries, prediction tests, and letterboxing and unpadding tests are organized. Done means the withoutBG ONNX model has an agreed task and weight packaging, integration coverage, documentation, and required notices.

Written by the indexing model from the issue text.

Description

enhancement
Search before asking
  • I have searched the Inference issues and found no similar feature requests.
Description

I'd like to add withoutBG Open Weights as a foundation alpha matting / background removal model to inference / inference_models.

Model: https://huggingface.co/withoutbg/withoutbg-openweights-onnx
I'm the author and trainer of this model.

What it is
  • Self-contained ONNX model (opset 18, ~455 MB, fp32)
  • Pipeline combines Depth Anything V2 (embedded in the graph) with a ConvNeXt-based matting head built on a DINOv3 ConvNeXt backbone
  • No trimap required. Takes an RGB image as input and produces a soft alpha matte
  • Fixed 448×448 input using letterboxing. A sidecar JSON documents the input/output names, tensor shapes, and SHA-256 checksum
I/O contract
Name Shape Dtype Range
Input rgb [1, 3, 448, 448] float32 [0, 1] NCHW
Output alpha [1, 1, 448, 448] float32 [0, 1]

Pre/post-processing is straightforward:

  1. Letterbox the longest side to 448.
  2. Pad to the top left.
  3. Run the ONNX model.
  4. Remove the padding.
  5. Resize the alpha matte back to the original image size.
  6. Optionally produce an RGBA cutout.
Proposed integration (v1)
  1. Add an ONNX model class to inference_models with from_pretrained, plus a registry entry under a new task such as image-matting or background-removal (happy to follow the maintainers' preference).
  2. Add a workflow block modeled after depth_estimation@v1 that outputs alpha (NUMPY_ARRAY_KIND) together with an optional cutout or visualization image.
  3. Add documentation, unit tests for letterboxing and unpadding, a prediction test, and the required license and third-party notices.
  4. Add an HTTP /infer adapter later if it makes sense.
Licensing
  • Model weights: Apache-2.0
  • Third-party components:
    • Depth Anything V2: Apache-2.0
    • DINOv3: Meta's DINOv3 license, consistent with how DINOv3 classification models are already documented in this repository
Questions
  1. What task type would you prefer for the registry?
  2. How would you prefer to package the weights and assign model IDs? Roboflow model package or direct download from Hugging Face?
  3. Any concerns about shipping a fused ONNX model that includes DINOv3 ConvNeXt components?

I'm happy to put together the implementation once we're aligned on the above.

Use case
  • Background removal (soft alpha, not a hard mask)
  • Product & portrait isolation where fine details matter (hair, edges) and instance / semantic segmentation is not enough
  • Image compositing (alpha + original -> cutout / new background)
Additional

See model outputs: https://withoutbg.com/open-model/results

Image
Are you willing to submit a PR?
  • Yes I'd like to help by submitting a PR!
Dominant language
Python
Stars
2.5k
Forks
340
Avg merge
2d 11h
Merged PRs (30d)
92

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/inference

All issues in roboflow/inference

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.