[ASK] Abnormal performance when training NPA algorithm
#1,260 opened on Dec 17, 2020
Repository metrics
- Stars
- (17,706 stars)
- PR merge metrics
- (Avg merge 6d 16h) (10 merged PRs in 30d)
Description
Description
Hello, an amazing work! Thanks for opensouring all code!
However, when i use NPA algorithm training on a self-made dataset, i find its abnormal performance.
That is, the training loss is decresing, and surprisely the group_auc metric is also decreasing.
The network's performance(group_auc) decrease even at the 2nd epoch, so i think this case is hard to be viewd as overfitting.
Train Settings:
epoch: we have tried 2, 5, 10 word_emb_dim: 1024, here we use the BERT pretrained model to do word embedding, so we adjusted this arg. other args: keep consistant with official setting
initial training loss: 1.6 initial group_auc: 0.54
Other Comments
i also have another problem. What is the meaning of the first evaulation result before training?
Look forward your reply, thanks~