piskvorky/gensim

WordEmbeddingsKeyedVectors.add() doesn't clear `vectors_norm`, causing `IndexError` on later `most_similar()`

Open

#2,532 创建于 2019年6月18日

在 GitHub 查看
 (17 评论) (0 反应) (0 负责人)Python (15,144 star) (4,349 fork)batch import
Hacktoberfestbugdifficulty easygood first issueimpact MEDIUMreach LOW

描述

As reported in a StackOverflow question/answer: https://stackoverflow.com/a/56641265/130288

An adapted version of the asker's minimal test case (which could become a unit test):

import numpy as np
from gensim.models.keyedvectors import WordEmbeddingsKeyedVectors

kv = WordEmbeddingsKeyedVectors(vector_size=3)
kv.add(entities=['a', 'b'],
       weights=[np.random.rand(3), np.random.rand(3)])
kv.most_similar('a')  # works

kv.add(entities=['c'], weights=[np.random.rand(3)])
kv.most_similar('c')  # fails with `IndexError`

Clearing the vectors_norm property (with either del or assignment-to-None) should be sufficient to trigger re-calculation upon the next most_similar().

贡献者指南

技术栈
pythonnumpy
领域
machine learning
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
pythonbasic numpyfamiliarity with gensim
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
60
研究方向
Look at the file `gensim/models/keyedvectors.py`, specifically the `add()` method and the `most similar()` method. The issue is that `add()` does not clear the cached `vectors norm` property, causing `most similar()` to fail with `IndexError` when called after adding new entities. The fix is to set `self.vectors norm = None` inside `add()` after modifying the vectors. The provided test case can be used to verify the fix. No maintainer clarification needed; the solution is straightforward.