Retire custom AdamWithDecay · facebookresearch/ParlAI#2232

(1 评论) (0 反应) (0 负责人)Python (10,371 star) (2,089 fork)batch import

AgentsCode QualityHelp WantedMinordonotreap

描述

We have a custom Adam optimizer that says it could be removed via pytorch 1.2:

The task is to remove it and upgrade to the official pytorch version.

The helper function minimizes the pain required, though I'm wondering if we'll need to upgrade some state dicts, so make sure you run long tests. https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert_ranker/helpers.py#L205

技术栈: pythonpytorch
领域: machine learningbackend
议题类型: refactor
难度: 3
预计时间: 1-2 days
活动状态: fresh
清晰度: mostly clear
前置要求: Familiarity with PyTorch optimizersUnderstanding of state dict migration
新手友好度: 40
研究方向: Examine the custom AdamWithDecay implementation at https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert ranker/helpers.py#L242 L350 and the helper function that minimizes pain. Replace the custom optimizer with torch.optim.Adam, ensuring any state dict differences are handled. Run long tests to verify correctness. Check if any other parts of the codebase depend on the custom implementation.