Clarification on [1] in swinunetr_pretrained README and training pipeline of ssl_pretrained_weights.pth
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 38/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- pytorch
- Domain
- documentation, machine-learning
Research direction
Start with the self_supervised_pretraining/swinunetr_pretrained/README.md and its reference [1], then compare the documented pipeline with research-contributions/SwinUNETR/Pretrain and its SSLHead. Use the listed ssl_pretrained_weights.pth checkpoint keys to trace which pipeline produced it. Done when the README identifies [1] and clearly states the checkpoint's training pipeline.
Written by the indexing model from the issue text.
Description
Hi MONAI team,
I’m trying to understand the origin of the pretrained weight:
ssl_pretrained_weights.pth (from MONAI-extra-test-data, ~719MB).
From the checkpoint structure, it contains:
- encoder.*
- decoder*
- out.conv.*
- encoder.mask_token
which looks like a full encoder-decoder model.
I found the following description in the tutorial README:
https://github.com/Project-MONAI/tutorials/blob/main/self_supervised_pretraining/swinunetr_pretrained/README.md
"The entire SwinUNETR model including encoder and decoder was trained end-to-end using self-supervised learning techniques as outlined in [1]."
However, it is unclear what [1] refers to in terms of actual training code or pipeline.
Could you please help clarify:
-
What exactly does reference [1] correspond to?
(Is it a specific paper, or an internal training implementation?) -
Was
ssl_pretrained_weights.pthtrained using the
research-contributions/SwinUNETR/Pretrain(SSLHead: rotation + contrastive + reconstruction),
or a different encoder-decoder / autoencoder-style pretraining pipeline?
A short clarification would already help a lot.
Thanks!
- Dominant language
- Jupyter Notebook
- Stars
- 2.5k
- Forks
- 803
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Project-MONAI/tutorials
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Project-MONAI/tutorials#2076 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 76/100
Project-MONAI/tutorials#2069 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Project-MONAI/tutorials#1704 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
Project-MONAI/tutorials#2071 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Project-MONAI/tutorials#2067 ·
All issues in Project-MONAI/tutorials
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Ecosystem: ClawMetry — the Qwen Code reader is now free and open source (follow-up to #9294 / #9338) Opencategory/integration priority/P3 scope/documentation status/ready-for-human type/feature-request
Difficulty 1/5 Under an hour Newbie friendliness 84/100