Question about the choice of decoder head in MoGe / MoGe-2

Open
#155 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Quiet

Research direction

No file, test, or entry point is named. Read the MoGe and MoGe-2 decoder implementations and any available paper or ablation material, then document the rationale, empirical comparisons, and efficiency trade-offs requested; done means a maintainer-confirmed explanation addressing the five questions.

Written by the indexing model from the issue text.

Description

Hi, thank you for releasing this excellent work and the codebase.

I have a question about the decoder design in MoGe and MoGe-2. From my understanding of the implementation, both models do not use a DPT-style decoder head. Instead, they adopt a custom lightweight convolutional head for dense prediction. Roughly speaking, this head seems to have significantly fewer parameters than a standard DPT head, around one third in my estimation.

Since DPT-style heads are widely used in monocular depth estimation and dense prediction tasks, I was wondering about the motivation behind this design choice.

Specifically, I would like to ask:

  1. Was the lightweight convolutional head chosen mainly for efficiency, such as reducing parameters, memory usage, or inference cost?
  2. Did you observe any empirical advantage of this head over a DPT-style head for point-map prediction or normal prediction?
  3. Is the DPT head less suitable for MoGe because the model predicts 3D point maps / metric geometry rather than only depth?
  4. Were there any internal ablation experiments comparing the proposed head with a DPT-style decoder head, even if they were not included in the paper?
  5. In your experience, would replacing the current decoder with a DPT-style head be likely to improve accuracy, or would it bring limited benefit compared with its extra computational cost?

I am asking because I am interested in extending MoGe-style point-map prediction to video geometric dense prediction, and the decoder-head design seems important for balancing accuracy and efficiency.

Thanks again for your great work!

Dominant language
JavaScript
Stars
3k
Forks
234
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/MoGe

All issues in microsoft/MoGe

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.