Problems in reproducing the RL fine-tuned results
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 25/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
调研方向
Start by reviewing generate.py and the train_actor.sh, generate_critic_scores.sh, and train_actor_rl.sh workflow described in the issue. Reproduce the reported 0.84 pass@5 result, then trace the inputs and default settings across the RL fine-tuning stage; done means documenting a verified configuration or a clearly isolated cause of the mismatch.
由索引模型根据 Issue 内容生成。
描述
Hi, thanks for open-sourcing your amazing work!
I have been trying to reproduce the RL fine-tuned results reported in the paper, but unfortunately, I am encountering some issues. Here is a brief overview of the steps I followed:
-
Fine-tuned the actor model with CE loss for 10 epochs with
train_actor.shand the CodeT5-NTP model. This fine-tuned model gives similar results to the paper (2.86 pass@5 compared to 2.90 in the paper) -
With some modifications to
generate.py, generated 20 candidate samples per problem (following the sample files given in the repo) and greedy baseline codes for the training set with the CE fine-tuned model. Theresultkey required for the correspondinggen_solutions.jsonandbaseline_solutions.jsonwas generated with this snippet. -
Generated the token level hidden states/critic scores with the released critic model through
generate_critic_scores.sh. -
RL-finetuning with the default hyperparameters present in
train_actor_rl.sh, the RL-finetuned model gives very degraded results. (0.84 pass@5)
I would greatly appreciate any suggestions you may have on hyperparameter choices or other settings that could help me reproduce the RL-finetuned results accurately.
Many thanks!
- 主要语言
- Python
- 星标
- 577
- 派生
- 69
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
salesforce/CodeRL 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 10/100
salesforce/CodeRL#65 ·
-
CodeRL模型谷歌云盘访问遭拒未关闭
难度 4/5 3-5 天 新手友好度 10/100
salesforce/CodeRL#64 ·
-
难度 5/5 一周以上 新手友好度 10/100
salesforce/CodeRL#63 ·
-
难度 3/5 1-2 天 新手友好度 32/100
salesforce/CodeRL#61 · 3 条评论 ·
-
难度 3/5 1-2 天 新手友好度 20/100
salesforce/CodeRL#60 ·
查看 salesforce/CodeRL 的全部 Issue
相似的 Issue
-
documentation
难度 2/5 1-3 小时 新手友好度 72/100
ianktoo/project-monitor#14 ·
-
security
难度 2/5 1-3 小时 新手友好度 84/100
btclib-org/btclib-wallet#478 ·
维护者通常 1 天内回复
-
security
难度 2/5 1-3 小时 新手友好度 75/100
btclib-org/btclib#2727 ·
维护者通常 1 天内回复
-
good first issue lead variant:container
难度 2/5 1-3 小时 新手友好度 76/100
radialmonster/orf-cleanroom#10 ·
维护者通常 1 天内回复
-
gdsa opencode: keep-alive cleanup trap fails with 'keepalive_pid: unbound variable'可能已有人在做 @darribas 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 85/100