How to comprehend the evaluation results?
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 20/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- documentation, machine-learning
Research direction
Start by locating where the evaluation results are defined or documented, then trace the meanings of "predicted_parse_error" and "exact". Clarify whether each value indicates parsing failure or SQL correctness, including why "exact" can be true, 0, or false; the issue is done when these output values are documented for users.
Written by the indexing model from the issue text.
Description
Hello，
when I get the evaluation results,I am not sure the meaning of "predicted_parse_error" and "exact".I guess when "predicted_parse_error" is true it means the model can’t produce a predicted sql,is that right?And I found "exact" has 3 possibilities:true,0 and false,I guess when "exact" is true ,it means the predicted sql is right.But what do 0 and false mean?
Thank you.
- Dominant language
- Python
- Stars
- 109
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from awslabs/gap-text2sql
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
awslabs/gap-text2sql#37 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
awslabs/gap-text2sql#35 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
awslabs/gap-text2sql#34 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
awslabs/gap-text2sql#33 ·
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
awslabs/gap-text2sql#32 ·
All issues in awslabs/gap-text2sql
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100