Anchors not being properly distributed across feature maps?

*Edit: based on the rpn.py documentation, it sounds like this may be intended behavior for RPN - in which case this is really more of an issue in faster_rcnn, in the way that its default anchor sizes are specified. RPN docs note that "sizes[i] and aspect_ratios[i] can have an arbitrary number of elements," but this doesn't seem to be totally true - it looks like sizes[i] must have the same number of elements for all feature maps.

https://github.com/pytorch/vision/blob/d6ee8757eca7b74b98e5f0d434a565eb7b1c410b/torchvision/models/detection/rpn.py#L122

This zip statement will result in anchors being split up across each feature map, as opposed to computing all anchors at each feature map. For example, if anchor sizes are defined as ((8,), (16,), (32,), (64,), (128,)), the first feature map will only be processed using anchors size 8, the second feature map size 16, etc. Specifying a number of anchor sizes that is different from the number of feature maps will result in a crash, since the number of anchors and output predictions will not match.