PetrochukM/PyTorch-NLP

Support loading fasttext model from custom file

Open

#61 aperta il 3 gen 2019

Vedi su GitHub
 (5 commenti) (0 reazioni) (0 assegnatari)Python (262 fork)batch import
enhancementgood first issuehelp wanted

Metriche repository

Star
 (2205 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

What if I want to use own pretrained fasttext model (or even commoncrawl model instead of standard wiki one)? E.g. look what they publish now: https://fasttext.cc/docs/en/crawl-vectors.html. Current FastText impl

    url_base = 'https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.{}.vec'
    aligned_url_base = 'https://s3.amazonaws.com/arrival/embeddings/wiki.multi.{}.vec'

    def __init__(self, language="en", aligned=False, **kwargs):
        if aligned:
            url = self.aligned_url_base.format(language)
        else:
url = self.url_base.format(language)

doesn't allow you to do such basic stuff.

Guida contributor