Turn near-duplicate score test into a property-based test
#944 aperta il 5 gen 2024
Metriche repository
- Star
- (7479 star)
- Metriche merge PR
- (Nessuna PR mergiata in 30 g)
Descrizione
After updating the near-duplicate scores, a test was added to ensure that near-duplicate examples have worse scores than non-near-duplicates. RIght now, the test only works on a small, toy dataset. Turning it into a property-based test would be ideal.
So that the test doesn't take too long, it may be a good idea to make a different embedding strategy that:
- Generates a random array
- Selects a handful of random examples to have near-duplicates. Append a new example that takes the original example with variable noise.
# append to existing feature array x[sample_ids] + small_random_noise - The embedding strategy should always generate "proper" near-duplicates that will get flagged. We don't want to test cases where there are no near-duplicates.
Below is the existing test case for comparison. We want to make sure that the issue scores created by the issue manager don't violate this property.
Originally posted by @elisno in https://github.com/cleanlab/cleanlab/pull/943#discussion_r1442401494