[ASK] Abnormal performance when training NPA algorithm
#1.260 geöffnet am 17. Dez. 2020
Repository-Metriken
- Stars
- (17.706 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 6T 16h) (10 gemergte PRs in 30 T)
Beschreibung
Description
Hello, an amazing work! Thanks for opensouring all code!
However, when i use NPA algorithm training on a self-made dataset, i find its abnormal performance.
That is, the training loss is decresing, and surprisely the group_auc metric is also decreasing.
The network's performance(group_auc) decrease even at the 2nd epoch, so i think this case is hard to be viewd as overfitting.
Train Settings:
epoch: we have tried 2, 5, 10 word_emb_dim: 1024, here we use the BERT pretrained model to do word embedding, so we adjusted this arg. other args: keep consistant with official setting
initial training loss: 1.6 initial group_auc: 0.54
Other Comments
i also have another problem. What is the meaning of the first evaulation result before training?
Look forward your reply, thanks~