Data loading in multi-lingual translation

Hello, I use the multi-lingual translation task and find some issues:

1. When I use the round_robin_dataset and multi_corpus_sampled_dataset for more than 20 language pairs, the data loading will take much more time in the functions of filter_by_size and batch_by_size. I find that in multi_corpus_sampled_dataset, the computation cost in calling the function of num_tokens is O(N^2). Could you provide some optimization for these two datasets?

2. The data loading will fail (RuntimeError: received 0 items of ancdata) when I use more language pairs, such as 28 language pairs. I find there is a daemon thread, which supports the buffer mechanism. When I close the buffer mechanism, the data loading can succeed. I do not figure out why this happens. Could you fix this issue? Thank you very much.