facebookresearch/ParlAI

Standardize batchifying image features

Open

#2,461 opened on Mar 11, 2020

View on GitHub
 (4 comments) (1 reaction) (0 assignees)Python (10,371 stars) (2,089 forks)batch import
H2 2020Help WantedSmalldonotreap

Description

Currently, ImageSeq2seqAgent and ImagePolyencoderAgent have two separate .batchify_image_features() methods for preparing image features for encoding. The main advantage of ImagePolyencoderAgent's implementation of this method is that the images are packed into a Tensor for data_parallel. It's worth testing whether ImageSeq2seqAgent really requires a separate .batchify_image_features() method now: if not, it should be removed, and ImagePolyencoderAgent.batchify_image_features() should be moved into TorchImageAgent.

Contributor guide

Tech stack
python
Domain
machine learningbackend
Issue type
refactor
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
ParlAI agentsPyTorch DataParallelImage feature batchification
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65
Research direction
Examine the implementation of batchify image features in both ImageSeq2seqAgent and ImagePolyencoderAgent within the ParlAI codebase. Determine if the differences are necessary due to architectural constraints or if the Polyencoder version can be generalized. If possible, move the Polyencoder's method to TorchImageAgent and remove the duplicate in Seq2seq. Refer to the discussion in issue #2461 for context.