facebookresearch/ParlAI

Retire custom AdamWithDecay

Open

#2,232 建立於 2019年12月2日

在 GitHub 查看
 (1 留言) (0 反應) (0 負責人)Python (10,371 star) (2,089 fork)batch import
AgentsCode QualityHelp WantedMinordonotreap

描述

We have a custom Adam optimizer that says it could be removed via pytorch 1.2:

https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert_ranker/helpers.py#L242-L350

The task is to remove it and upgrade to the official pytorch version.

The helper function minimizes the pain required, though I'm wondering if we'll need to upgrade some state dicts, so make sure you run long tests. https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert_ranker/helpers.py#L205

貢獻者指南

技術棧
pythonpytorch
領域
machine learningbackend
議題類型
refactor
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-2 days
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
fresh
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
mostly clear
前置要求
Familiarity with PyTorch optimizersUnderstanding of state dict migration
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
40
研究方向
Examine the custom AdamWithDecay implementation at https://github.com/facebookresearch/ParlAI/blob/3a5de86667816abbdb0f08049c86d1e2c3aff03e/parlai/agents/bert ranker/helpers.py#L242 L350 and the helper function that minimizes pain. Replace the custom optimizer with torch.optim.Adam, ensuring any state dict differences are handled. Run long tests to verify correctness. Check if any other parts of the codebase depend on the custom implementation.