Evaluation loss becomes constant

Hi!

I am testing out unsloth to fine tune llama 3.1 8B instruct and following your notebook here.

One exception is that I have added an eval set. What is really strange is that the eval loss locks up to a specific value after around 300 steps. I mean down to the last decimal, not just flattening out. The training loss looks fine and as expected.

I have changed many parameters and tried different things but it always happens. Any idea on what can cause this?