[ASK] Abnormal performance when training NPA algorithm
#1260 aperta il 17 dic 2020
Metriche repository
- Star
- (17.706 star)
- Metriche merge PR
- (Merge medio 6g 16h) (10 PR mergiate in 30 g)
Descrizione
Description
Hello, an amazing work! Thanks for opensouring all code!
However, when i use NPA algorithm training on a self-made dataset, i find its abnormal performance.
That is, the training loss is decresing, and surprisely the group_auc metric is also decreasing.
The network's performance(group_auc) decrease even at the 2nd epoch, so i think this case is hard to be viewd as overfitting.
Train Settings:
epoch: we have tried 2, 5, 10 word_emb_dim: 1024, here we use the BERT pretrained model to do word embedding, so we adjusted this arg. other args: keep consistant with official setting
initial training loss: 1.6 initial group_auc: 0.54
Other Comments
i also have another problem. What is the meaning of the first evaulation result before training?
Look forward your reply, thanks~