PetrochukM/PyTorch-NLP

Support loading fasttext model from custom file

Open

#61 创建于 2019年1月3日

在 GitHub 查看
 (5 评论) (0 反应) (0 负责人)Python (2,205 star) (262 fork)batch import
enhancementgood first issuehelp wanted

描述

What if I want to use own pretrained fasttext model (or even commoncrawl model instead of standard wiki one)? E.g. look what they publish now: https://fasttext.cc/docs/en/crawl-vectors.html. Current FastText impl

    url_base = 'https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.{}.vec'
    aligned_url_base = 'https://s3.amazonaws.com/arrival/embeddings/wiki.multi.{}.vec'

    def __init__(self, language="en", aligned=False, **kwargs):
        if aligned:
            url = self.aligned_url_base.format(language)
        else:
url = self.url_base.format(language)

doesn't allow you to do such basic stuff.

贡献者指南

技术栈
pythonpytorch
领域
backendmachine learning
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
Python basicsPyTorch basicsFastText format
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
75
研究方向
Examine the current FastText class implementation (likely in 'pytorchnlp/fasttext.py'). Add a parameter to init for a custom file path. Update the URL logic to use the provided path if given, else fall back to defaults. Ensure backward compatibility. See issue comments for additional context.