pytorch/text

Vocab vectors using complete pretrained-embedding?

Open

Aperta il 12 ott 2018

Vedi su GitHub
 (6 commenti) (0 reazioni) (0 assegnatari)Python (3396 star) (822 fork)batch import
enhancementhelp wanted

Descrizione

I am new to pytorch and nlp. I have a question when I tried to build a model.

Since my training dataset is not so big, the size of its vocab is relatively small (around 5000). However, I want to deal with any other user input which could be out of this vocabulary.

The problem is, in the model I trained, the embedding layer's weight is based on the vectors of the field, not the whole word2vec pretrained embeddings. So I cannot modified it after the training is done.

I wondered is there any better approach to do it? Thanks in advance!

Guida contributor

Tech stack
python
Dominio
machine learning
Tipo issue
feature
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
half day
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
clear
Prerequisiti
PyTorch basicsword embeddings
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
50
Direzione di ricerca
Investigate how PyTorch's torchtext handles vocabulary and embedding layers. Look into existing discussion on this issue (e.g., comments on GitHub issue #446) and evaluate if a utility function for loading full pretrained embeddings is feasible. Consider adding a method to allow users to specify a pretrained embedding source that maps tokens not in the training vocab to their vectors from a larger set.