Add diminishing returns for notes in Solr scoring

未关闭
#412 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
冷清
技术栈
python
领域
search

调研方向

Start by locating the Solr query configuration and scoring tests, then compare the scores with note search enabled and disabled. Explore how repeated matching notes affect document scores; done means the first few note matches still contribute while additional matches have diminishing influence.

由索引模型根据 Issue 内容生成。

描述

planning solr

For logged in premium users, they have access to note search.
An example query:
https://www.documentcloud.org/documents/?q=Higganum+complaint

The first few documents are rated so highly not because they are super relevant in content, but because they have a LOT of notes with the word complaint in them. Did testing in the Django shell with my user (which has note search enabled) and those without and the resulting scores for the top 5 documents Solr returns are as follows:

WITHOUT notes:
  id: 2704406 score: 51.575443
  id: 3535180 score: 45.707706
  id: 336686 score: 44.942238
  id: 26948698 score: 41.152092
  id: 6953986 score: 41.038723

WITH notes:
  id: 321087 score: 397.05875
  id: 3034115 score: 186.30943
  id: 28167531 score: 147.98016
  id: 23789873 score: 147.8687
  id: 1369433 score: 132.84674

When looking deeper into the explanation:

doc 321087 has 106 notes matching 'complaint' in title:
note: 47030 title: Telephone-related complaint score: 3.3627408
note: 47031 title: Telephone-related complaint score: 3.3627408
note: 47032 title: Telephone-related complaint score: 3.3627408
note: 47033 title: Telephone-related complaint score: 3.3627408
note: 47036 title: Telephone-related complaint score: 3.3627408

We should add some kind of diminishing return mechanism here so the first few note hits impact score, but the returns diminish with too many

主要语言
Python
星标
55
派生
10
平均合并
1 天 13 小时
30 天内合并 PR
22

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

MuckRock/documentcloud 的其他 Issue

查看 MuckRock/documentcloud 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。