Opt-in API that has a cached `ThreadpoolController`
维护者通常 1 天内回复
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 25/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- python
- 领域
- performance
调研方向
The proposal is a top-level get_cached_controller(force_new=False) API that caches a controller and rebuilds it when the number of loaded modules in sys.modules changes. Work is already under way in linked pull request #257, so read that PR and the three-comment thread before starting, to see what is still undecided. Done means the cache invalidates on new imports and force_new=True always rebuilds, and that behaviour is covered by tests.
由索引模型根据 Issue 内容生成。
描述
Creating a controller is still expensive on Linux at least, so for libraries that do this a lot a cached version that notices when libraries change might be useful.
My thought is a top-level API get_cached_controller(force_new=False), that checks if e.g. the length of sys.modules has changed. If yes, it creates (and caches) a new controller, otherwise it returns a cached one. With force_new=True it always invalidates.
Application authors would presumably know that there's no other mechanism where new libraries might get loaded.
I would like to use this for
joblib, on the theory that:
- Joblib never had this oversubscription mitigation before.
- This will catch the vast majority of cases where new libraries get added.
- In the rare cases it doesn't catch a loaded library, it's not a bug, it's just a performance hit... and again, joblib never solved this before anyway so those users are no worse off.
- 主要语言
- Python
- 星标
- 434
- 派生
- 38
- 平均合并
- 2 天 9 小时
- 30 天内合并 PR
- 7
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
joblib/threadpoolctl 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 35/100
joblib/threadpoolctl#272 · 7 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 25/100
joblib/threadpoolctl#271 ·
维护者通常 1 天内回复
-
Make it possible to inspect active/spin wait behavior of LLVM/Intel OpenMP via `kmp_get_blocktime`未关闭
难度 3/5 1-2 天 新手友好度 42/100
joblib/threadpoolctl#270 ·
维护者通常 1 天内回复
-
难度 4/5 1-2 天 新手友好度 32/100
joblib/threadpoolctl#264 · 2 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 22/100
joblib/threadpoolctl#261 · 3 条评论 ·
维护者通常 1 天内回复
查看 joblib/threadpoolctl 的全部 Issue
相似的 Issue
-
Use mhctools predictor_required=False for saved-result rescoring可能已有人在做 @iskandr 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 79/100
-
难度 1/5 1 小时以内 新手友好度 85/100
521xueweihan/HelloGitHub#4051 · 1 条评论 ·
-
bug documentation
难度 2/5 1-3 小时 新手友好度 78/100
awslabs/visual-asset-management-system#514 ·
维护者通常 1 天内回复
-
backend bug spec:REQ-053
难度 2/5 1-3 小时 新手友好度 66/100
nolte/kamerplanter#2547 ·
维护者通常 1 天内回复