reworkd/AgentGPT

✨ Investigate the best similarity score threshold to remove duplicate tasks

Open

#729 创建于 2023年6月6日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)TypeScript (34,594 star) (9,446 fork)batch import
enhancementhelp wanted

描述

When we generate tasks, we filter tasks that have a similarity score that is too close to existing tasks in the vector database

similar_tasks = memory.get_similar_tasks(
    task, score_threshold=0.95  # TODO: Once we use ReAct, revisit
)

This is done with the help of the code above. Arbitrarily, we use 0.95. Even with this, the task may not be related.

On the other hand, there may be very related / duplicated tasks that have a score that is less than this.

This ticket is tasked with investigating what the best value for this threshold is, or to use some other means of calculating similarity for this given case.

贡献者指南

技术栈
typescript
领域
aidatabackend
议题类型
research
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
similarity metricsvector databasesAgentGPT codebase
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
60
研究方向
Investigate the function memory.get similar tasks in the codebase to understand current implementation. Experiment with different score threshold values (e.g., 0.9, 0.8, 0.7) and evaluate the impact on task deduplication using a test dataset. Analyze the trade offs between false positives and false negatives and propose a recommended threshold or alternative similarity approach.
✨ Investigate the best similarity score threshold to remove duplicate tasks · reworkd/AgentGPT#729 | Good First Issue