`get_raw_interactions()` NaN assumptions and edge case
まだ誰も着手していません。
評価
- 難易度
- 2/5
- 見積もり時間
- 1〜3時間
- 初心者へのやさしさ
- 35/100
調査の方向性
The issue traces the behavior through Dataset.get_raw_interactions(), Interactions.to_external(), identifier conversion, and get_from_series_by_index() in rectools/utils/indexing.py. Read those entry points and the related tests to see how strict is handled and how raw missing IDs behave. Done means the reported NaN case has defined behavior and a regression test; the issue has had no comment for 564 days.
索引モデルが issue の本文から書いたものです。
説明
What happened?
Hello! Great work on the library! I recently found an edge case in the get_raw_interactions() calls, which may or may not be intended, so I'm opening an issue for discussion.
Context
Namely, the get_raw_interactions() method is a couple of wrappers for to the convert_to_external() calls from each identifier, which uses get_from_series_by_index().
get_from_series_by_index() seems to have an explicit assumption that NaNs in the resulting dataframe are caused by the reindex() operation. The operation introduces NaNs when there is a mismatch between item IDs and user IDs:
https://github.com/MobileTeleSystems/RecTools/blob/main/rectools/utils/indexing.py#L123-L127
Edge Case
The reindex() operation isn't the only time NaNs can be introduced, though. For example, if a Dataset has a NaN by itself in its raw form - this exception is raised anyways. For example:
>>> import rectools
>>> import pandas as pd
>>> from rectools.dataset import Dataset
>>> data = {
... 'user_id': [1, 2, 3, None, 5, 1, 2, 3, 4, 5], # There is a None in the User IDs
... 'item_id': [101, 102, 103, 104, 105, 106, 107, 108, 109, 110],
... 'rating': [5, 3, 4, 2, 1, 4, 5, 3, 2, 1]
... }
>>> df = pd.DataFrame(data)
>>> df['weight'] = 1
>>> df['datetime'] = pd.to_datetime('2025-02-25')
>>> dataset = Dataset.construct(df)
>>> dataset.get_raw_interactions()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/david.landup/.pyenv/versions/3.11.10/lib/python3.11/site-packages/rectools/dataset/dataset.py", line 248, in get_raw_interactions
return self.interactions.to_external(self.user_id_map, self.item_id_map, include_weight, include_datetime)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/david.landup/.pyenv/versions/3.11.10/lib/python3.11/site-packages/rectools/dataset/interactions.py", line 181, in to_external
Columns.User: user_id_map.convert_to_external(self.df[Columns.User].values),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/david.landup/.pyenv/versions/3.11.10/lib/python3.11/site-packages/rectools/dataset/identifiers.py", line 221, in convert_to_external
result = get_from_series_by_index(self.to_external, internal, strict, return_missing)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/david.landup/.pyenv/versions/3.11.10/lib/python3.11/site-packages/rectools/utils/indexing.py", line 127, in get_from_series_by_index
raise KeyError("Some indices do not exist")
KeyError: 'Some indices do not exist'
While get_from_series_by_index() itself has a very useful strict mode that can be turned off, the get_raw_interactions() method doesn't allow passing this argument down to the external export calls.
Workaround
A simple workaround is to simply re-implement the constituent calls and pass strict=False in a utility module or patch the lib. However, it would be much nicer developer UX if we can simply pass it through the call natively.
Proposal
One backwards-compatible way to allow for this edge case would be to allow **kwargs in the get_raw_interactions() method and pass them to the underlying calls.
Alternatively, it would be nice UX to also raise which indices are missing or provide a bit of extra context in the error message to alleviate part of the debugging.
Thoughts?
Expected behavior
No response
Relevant logs and/or screenshots
No response
Operating System
MacOS
Python Version
3.11
RecTools version
0.8.0
- 主要言語
- Python
- スター
- 457
- フォーク
- 55
- PR マージ指標
- 30日以内にマージされた PR はありません
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
MTSWebServices/RecTools のほかの issue
-
good first issue
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
MTSWebServices/RecTools#330 ·
-
Is it not possible to `load` transformer model on cpu only対応中かも @maxteneff が 22 日前に担当しました。 オープンbug good first issue
難易度 3/5 1〜2日 初心者へのやさしさ 35/100
MTSWebServices/RecTools#286 · コメント 3 件 ·
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
MTSWebServices/RecTools#249 ·
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
MTSWebServices/RecTools#243 ·
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
MTSWebServices/RecTools#242 ·
MTSWebServices/RecTools の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 83/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 87/100
メンテナーはふだん 7 日以内に返信
-
area: web priority: P1 story
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
nfiasco/Quant-Time-Machine#151 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
autowarefoundation/autoware_carla_scenario#2019 ·
メンテナーはふだん 1 日以内に返信
-
docs pydanty:is-working
難易度 1/5 1〜3時間 初心者へのやさしさ 85/100
pydantic/pydantic-ai#12892 ·
メンテナーはふだん 1 日以内に返信