Some Questions about Quality Model~
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 25/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Quiet
- Domain
- ai, machine-learning
Research direction
Start with Appendix A2.1 and the description of the quality scorer's training and inference pipeline. The issue is resolved when the maintainers clarify whether the fixed prompt is used during training and whether prompting is required with the pretrained Llama backbone.
Written by the indexing model from the issue text.
Description
Hi there! Thanks for your great work. I have a few questions regarding your custom-trained code quality scorer model.
The paper mentions that you adopted a Llama-series pretrained model as the backbone. However, in Appendix A2.1 about the evaluation prompt, it states:
"It remains consistent throughout the entire pipeline, from collecting ground-truth data to training the quality scorer and applying it across all GitHub data during inference."
I would like to confirm:
Does this mean you appended the fixed prompt to code samples during the training phase of the quality model?
Additionally, since the base Llama model is only pretrained and has not undergone instruction tuning, is it necessary to feed a task-specific prompt together with code inputs for quality model training?
Thanks a lot for your clarification!
- Dominant language
- No language data
- Stars
- 759
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ByteDance-Seed/Seed-Coder
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
ByteDance-Seed/Seed-Coder#24 ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
ByteDance-Seed/Seed-Coder#23 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
ByteDance-Seed/Seed-Coder#22 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
ByteDance-Seed/Seed-Coder#21 ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
ByteDance-Seed/Seed-Coder#20 ·
All issues in ByteDance-Seed/Seed-Coder
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
code-yeongyu/senpi#2456 · 1 comment ·
Maintainers usually reply within 1 day
-
python triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
microsoft/semantic-kernel#14512 · 1 comment ·
Maintainers usually reply within 4 days
-
discussions enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
code-yeongyu/oh-my-openagent#9290 · 1 comment ·
Maintainers usually reply within 1 day
-
bug bughunt good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
rajveer43/VeloxQuant-MLX#656 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day