Is there a method to directly evaluate my trasnlated code snippets?
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- bash, python
- Domain
- machine-learning
Research direction
Start with codegen_sources/model/train.py and inspect check_data_params in codegen_sources/model/src/data/loader.py, then review eval.sh around line 17. Re-run the documented command against the downloaded test set and distinguish the missing .pth files from the shell error. Done means the evaluation command completes its initial data checks without either reported failure.
Written by the indexing model from the issue text.
Description
When I run the evaluation script you listed in readme:
MODEL=""
python codegen_sources/model/train.py \
--exp_name transcoder_eval \
--dump_path '<DUMP_PATH>' \
--data_path './data/test_dataset' \ #here I the validation set you provided in https://dl.fbaipublicfiles.com/transcoder/test_set/transcoder_test_set.zip
--bt_steps 'python_sa-java_sa-python_sa,java_sa-python_sa-java_sa,python_sa-cpp_sa-python_sa,java_sa-cpp_sa-java_sa,cpp_sa-python_sa-cpp_sa,cpp_sa-java_sa-cpp_sa' \
--encoder_only False \
--n_layers 0 \
--n_layers_encoder 6 \
--n_layers_decoder 6 \
--emb_dim 1024 \
--n_heads 8 \
--lgs 'cpp_sa-java_sa-python_sa' \
--max_vocab 64000 \
--gelu_activation false \
#--roberta_mode false \
--amp 2 \
--fp16 true \
--tokens_per_batch 3000 \
--max_batch_size 128 \
--eval_bleu true \
--eval_computation true \
--has_sentence_ids "valid|para,test|para" \
--generate_hypothesis true \
--save_periodic 1 \
--reload_model "$MODEL,$MODEL" \
--reload_encoder_for_decoder false \
--eval_only true \
--n_sentences_eval 1500
ERROR BELOW:
adding to path /home/yz/CodeGen 2023-05-26 03:01:30.703702: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-05-26 03:01:30.836176: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0. 2023-05-26 03:01:31.312719: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2023-05-26 03:01:31.312770: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2023-05-26 03:01:31.312777: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. adding to path /home/yz/CodeGen Using /home/yz/data/CodeGen/code_evaluation/tmp_tests_folder for temporary files. adding to path /home/yz/CodeGen No efficient attention. adding to path /home/yz/CodeGen adding to path /home/yz/CodeGen ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/train.cpp_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/valid.cpp_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/test.cpp_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/train.java_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/valid.java_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/test.java_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/train.python_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/valid.python_sa.pth not found ERROR - 05/26/23 03:01:32 - 0:00:00 - ./data/test_dataset/test.python_sa.pth not found Traceback (most recent call last): File "/home/yz/CodeGen/codegen_sources/model/train.py", line 1016, in check_data_params(params) File "/home/yz/CodeGen/codegen_sources/model/src/data/loader.py", line 582, in check_data_params assert all( AssertionError: [['./data/test_dataset/train.cpp_sa.pth', './data/test_dataset/valid.cpp_sa.pth', './data/test_dataset/test.cpp_sa.pth'], ['./data/test_dataset/train.java_sa.pth', './data/test_dataset/valid.java_sa.pth', './data/test_dataset/test.java_sa.pth'], ['./data/test_dataset/train.python_sa.pth', './data/test_dataset/valid.python_sa.pth', './data/test_dataset/test.python_sa.pth']] eval.sh: line 17: --amp: command not found
How to solve this problem?
Thanks!
- Dominant language
- Python
- Stars
- 777
- Forks
- 144
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from facebookresearch/CodeGen
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
facebookresearch/CodeGen#109 ·
-
Hash function Open
Difficulty 3/5 1-2 days Newbie friendliness 42/100
facebookresearch/CodeGen#108 ·
-
Dataset issues Open
Difficulty 3/5 1-2 days Newbie friendliness 45/100
facebookresearch/CodeGen#107 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
facebookresearch/CodeGen#106 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
facebookresearch/CodeGen#105 ·
All issues in facebookresearch/CodeGen
Similar issues
-
bug priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
CyberAgent/psd2svg#436 ·
-
area/install-update comp/cli comp/desktop P3 sweeper:risk-compatibility type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
NousResearch/hermes-agent#122386 · 1 comment ·
-
ai-generated
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vllm-project/production-stack#1105 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100