A way to specify policies for a specific backend
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
Read the existing controller API and the OpenBLASController implementation, then review how set_num_threads applies limits. The issue presents alternative policy hooks but does not choose a design; first determine which extension point fits the stated backend policies. Done means a selected design supports the intended policies, with tests covering the behavior.
由索引模型根据 Issue 内容生成。
描述
We have discussed multiple potential examples of having specific policies for specific libraries' backends, mostly OpenBLAS+pthreads:
- Limit to just 1 core (i.e. inline in current thread) or all cores, don't set any other limits but those two. These are really the most common ones you'd want for a shared process-level thread pool. "All cores" is a concept outside of threadpoolctl's scope, though!
- Only allow setting limits on the main thread, basically: https://github.com/joblib/joblib/pull/1871#discussion_r4340880131
The current API doesn't have great ergonomics for this though.
One approach is to push policies down into threadpoolctl. But that seems difficult, e.g. "all cores" example above (until Python grows a cpu count API that knows about cgroups, at least).
Another approach is to allow libraries (really, joblib) to hook in and define their own policies.
Potential designs
Replace default controller class with custom controller class
from joblib import cpu_count
from threadpoolctl import OpenBLASController, replace_controller
class JoblibOpenBLASController(OpenBLASController):
def set_num_threads(self, num_cores):
if self.threading_layer == "openmp":
return super().set_num_threads(num_cores)
cpus = cpu_count()
if num_cores >= 0.75 * cpus:
num_cores = cpus
else
num_cores = 1
return super().set_num_threads(num_cores)
replace_controller(OpenBLASController, JoblibOpenBLASController)
Register custom policy just for set_num_threads
from joblib import cpu_count
from threadpoolctl import OpenBLASController
def transform_num_threads(controller, num_cores):
if controller.threading_layer == "openmp":
return num_cores
cpus = cpu_count()
if num_cores >= 0.75 * cpus:
return cpus
else
return 1
return super().set_num_threads(num_cores)
OpenBLASController.set_transform_num_threads_before_setting_hook(transform_num_threads)
... other ideas?
- 主要语言
- Python
- 星标
- 434
- 派生
- 38
- 平均合并
- 2 天 14 小时
- 30 天内合并 PR
- 8
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
joblib/threadpoolctl 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 25/100
joblib/threadpoolctl#271 ·
维护者通常 1 天内回复
-
Make it possible to inspect active/spin wait behavior of LLVM/Intel OpenMP via `kmp_get_blocktime`未关闭
难度 3/5 1-2 天 新手友好度 42/100
joblib/threadpoolctl#270 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 22/100
joblib/threadpoolctl#261 · 3 条评论 ·
维护者通常 1 天内回复
-
Opt-in API that has a cached `ThreadpoolController`可能已有人在做 @itamarst 于 29 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 25/100
joblib/threadpoolctl#254 · 3 条评论 ·
维护者通常 1 天内回复
-
Speed up usage of threadpoolctl可能已有人在做 @itamarst 于 36 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 18/100
joblib/threadpoolctl#246 ·
维护者通常 1 天内回复
查看 joblib/threadpoolctl 的全部 Issue
相似的 Issue
-
[Bug]: Revit selector reads process ids 2000-2099 as years (flaky AgentClientTests.AYearOrAProcessIdSelectsTheHost)可能已有人在做 @Wurschdhaud 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 83/100
pyrevitlabs/pyRevit#3817 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 85/100
521xueweihan/HelloGitHub#4074 ·
-
follow-up
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
Helm chart: allow setting ingressClassName on the Ingress可能已有人在做 @commarla 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 85/100
pgadmin-org/pgadmin4#10658 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100