Option to redraw negatives that coincide with the target in `CatalogUniformSampler`

Open Beginner friendly
  • Is something wrong?
#330 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch
Domain
machine-learning

Research direction

Start with rectools/models/nn/transformers/negative_sampler.py and read CatalogUniformSampler.get_negatives; compare its collision handling with _NegativeSampler in rectools/fast_transformers/unisrec/model.py. Add the opt-in behavior through negative_sampler_kwargs, keeping the default behavior unchanged, and check the sampler's unit tests. Done means colliding negatives are redrawn when enabled and existing behavior is preserved by default.

Written by the indexing model from the issue text.

Description

good first issue
Feature Description

Add an opt-in parameter to CatalogUniformSampler (for example avoid_positive_collisions: bool = False) that redraws sampled negatives equal to the target item at the same position. It would be enabled through the existing negative_sampler_kwargs:

SASRecModel(loss="gBCE", n_negatives=256, negative_sampler_kwargs={"avoid_positive_collisions": True})

The default would keep the current behaviour, so existing configs and seeded results do not change.

Why this feature?

CatalogUniformSampler.get_negatives (rectools/models/nn/transformers/negative_sampler.py) samples negatives uniformly from all catalog items (as its docstring says) using torch.randint, without excluding the target. With BCE, gBCE or sampled_softmax, the same item can then be scored as both the positive and a negative in one loss term.

The UniSRec model added in 0.19.0 already handles this: _NegativeSampler in rectools/fast_transformers/unisrec/model.py redraws colliding negatives once. The transformer models in rectools.models.nn (SASRec, BERT4Rec, HSTU) have no such option, although users can subclass TransformerNegativeSamplerBase to get it.

The collision probability per negative is 1 / n_items. With n_negatives=256:

catalog size share of negatives equal to the target share of positions with at least one such negative
3,416 (ML-1M as used in the SASRec/gSASRec papers) 0.03% 7.2%
500 0.2% 40%
100 1.0% 92%

On a synthetic dataset with 100 items (2,000 users, 22 interactions each), SASRecModel with loss="sampled_softmax" and n_negatives=256 reached NDCG@10 of about 0.127 (RecTools NDCG(k=10), default settings). Redrawing colliding negatives until none were left changed this by -0.0009 on average over 3 seeds, which is within run-to-run noise. So I don't expect a measurable metric gain; the aim is consistency with UniSRec and sampling negatives the way the gBCE sampling rate alpha = n_negatives / (n_items - 1) already assumes.

Additional context

I have a small patch ready (one redraw pass, same as _NegativeSampler, plus unit tests and a changelog entry) and can open a PR if this is welcome.

Dominant language
Python
Stars
457
Forks
55
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MTSWebServices/RecTools

All issues in MTSWebServices/RecTools

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.