huggingface/transfer-learning-conv-ai

RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710

Open

#12 建立於 2019年6月4日

在 GitHub 查看
 (3 留言) (0 反應) (0 負責人)Python (429 fork)batch import
good first issue

倉庫指標

Star
 (1,700 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

I'm playing around with this wonderful code but I'm running into a curious issue when I try to train the model with my own data.

I replicated the personachat_self_original.json file structure and added my own data. I deleted dataset_cache_OpenAIGPTTokenizer file but when I try to train, I get this error:

INFO:train.py:Pad inputs and convert to Tensor
Traceback (most recent call last):
  File "train.py", line 252, in <module>
    train()
  File "train.py", line 164, in train
    train_loader, val_loader, train_sampler, valid_sampler = get_data_loaders(args, tokenizer)
  File "train.py", line 97, in get_data_loaders
    tensor = tensor.view((-1, datasets[dataset_name]["n_candidates"]) + tensor.shape[1:])
RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710

I have triple checked that my dataset follows the same structure but I can't figure out why the training script doesn't like it.

Any ideas why this is happening?

貢獻者指南