[ASK] Abnormal performance when training NPA algorithm

Description

Hello, an amazing work! Thanks for opensouring all code!

However, when i use NPA algorithm training on a self-made dataset, i find its abnormal performance.

That is, the training loss is decresing, and surprisely the group_auc metric is also decreasing.

The network's performance(group_auc) decrease even at the 2nd epoch, so i think this case is hard to be viewd as overfitting.

Train Settings:

epoch: we have tried 2, 5, 10 word_emb_dim: 1024, here we use the BERT pretrained model to do word embedding, so we adjusted this arg. other args: keep consistant with official setting

initial training loss: 1.6 initial group_auc: 0.54

Other Comments

i also have another problem. What is the meaning of the first evaulation result before training?

Look forward your reply, thanks~