Training MLM with reload model as TransCoder_model_1 on csharp monolingual data generated from pre_processing

Start with the MLM reload step and the TransCoder_model_1.pth loading logs. Compare the checkpoint's supported languages, vocabulary, position-embedding size, and reported missing parameters with the csharp_monolingual run. Done means determining whether these warnings and early stopping are expected and identifying the relevant configuration or checkpoint mismatch.

We have pre-processed our data in csharp and ruby, and obtained their monolingual data and after that we are trying to run the MLM step by reloading TransCoder_model_1, but in the logs we see these warnings and also the accuracy for the run is pretty low and the training stops after 10 epochs as there is no improvement. Then, we've tried givign only csharp as the language and it's the same. Is the below behaviour expected while training?

INFO - 04/05/23 15:15:35 - 0:00:05 - ============ Model Reloading
INFO - 04/05/23 15:15:35 - 0:00:05 - Reloading encoder from TransCoder_model_1.pth ...
WARNING - 04/05/23 15:15:43 - 0:00:13 - When reloading word embeddings, could not find embeddings for 29436 words: ['CLASS_0', 'CLASS_1', 'CLASS_2', 'CLASS_3', 'CLASS_4', '...', '経', '者の@@', '要素の@@', '見た@@', '鉛@@']... Initializing them to < unk >.
WARNING - 04/05/23 15:15:44 - 0:00:14 - No match found for lang csharp_monolingual csharp_monolingual in dict_keys(['cpp_sa', 'java_sa', 'python_sa']). Initializing randomly.
WARNING - 04/05/23 15:15:44 - 0:00:14 - The size of position embeddings in current model is 2048, the size of reloaded is 1024. need to repeat last positions 1024 times.
INFO - 04/05/23 15:15:44 - 0:00:14 - Reloading decoders from TransCoder_model_1.pth ...
WARNING - 04/05/23 15:15:44 - 0:00:15 - When reloading word embeddings, could not find embeddings for 29436 words: ['CLASS_0', 'CLASS_1', 'CLASS_2', 'CLASS_3', 'CLASS_4', '...', '経', '者の@@', '要素の@@', '見た@@', '鉛@@']... Initializing them to < unk >.
WARNING - 04/05/23 15:15:45 - 0:00:15 - No match found for lang csharp_monolingual csharp_monolingual in dict_keys(['cpp_sa', 'java_sa', 'python_sa']). Initializing randomly.
WARNING - 04/05/23 15:15:45 - 0:00:15 - The size of position embeddings in current model is 2048, the size of reloaded is 1024. need to repeat last positions 1024 times.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter layer_norm15.0.weight not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter layer_norm15.0.bias not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter encoder_attn.0.q_lin.weight not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter encoder_attn.0.q_lin.bias not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter encoder_attn.0.k_lin.weight not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter encoder_attn.0.k_lin.bias not found.
WARNING - 04/05/23 15:15:45 - 0:00:15 - Parameter encoder_attn.0.v_lin.weight not found.