huggingface/transfer-learning-conv-ai

RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710

Open

#12 opened on 2019年6月4日

GitHub で見る
 (3 comments) (0 reactions) (0 assignees)Python (429 forks)batch import
good first issue

Repository metrics

Stars
 (1,700 stars)
PR merge metrics
 (30d に merged PR はありません)

説明

I'm playing around with this wonderful code but I'm running into a curious issue when I try to train the model with my own data.

I replicated the personachat_self_original.json file structure and added my own data. I deleted dataset_cache_OpenAIGPTTokenizer file but when I try to train, I get this error:

INFO:train.py:Pad inputs and convert to Tensor
Traceback (most recent call last):
  File "train.py", line 252, in <module>
    train()
  File "train.py", line 164, in train
    train_loader, val_loader, train_sampler, valid_sampler = get_data_loaders(args, tokenizer)
  File "train.py", line 97, in get_data_loaders
    tensor = tensor.view((-1, datasets[dataset_name]["n_candidates"]) + tensor.shape[1:])
RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710

I have triple checked that my dataset follows the same structure but I can't figure out why the training script doesn't like it.

Any ideas why this is happening?

コントリビューターガイド