Allow arbitrary number of input channels in ResNet encoder (not only RGB)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by locating the ResNet encoder construction and the code that applies channel-wise mean and standard deviation; the issue does not name specific files or tests. Review how the existing three-channel model is built and determine how arbitrary input channels, pretrained weights, and normalization should be represented. Done means users can construct the encoder with additional channels while preserving pretrained RGB behavior.
Written by the indexing model from the issue text.
Description
With https://github.com/mapbox/robosat/pull/46 we are changing our model architecture from training the encoder and decoder from scratch to using a pre-trained ResNet for the encoder. The pre-trained ResNet uses three channels (RGB) for the input layer through.
We need to be able to add arbitrary channels, say, RGB + water mask + elevation + lidar. To to this we need to construct a wrapper module architecture extending the ResNet architecture, copying weights over, and initializing the new channels with zero. In addition the channel-wise mean and std dev needs to be adapted.
Tasks
- Figure out how to extend the ResNet input channels
- Figure out how to copy over the pre-trained ResNet parts
- Let users construct a model with arbitrary channels
- Adapt mean and std dev
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 385
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mapbox/robosat
-
Difficulty 1/5 Under an hour Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
-
Difficulty 4/5 3-5 days Newbie friendliness 32/100
-
Difficulty 3/5 1-2 days Newbie friendliness 30/100
-
Bringin own data Open
Difficulty 5/5 Over a week Newbie friendliness 20/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100