`RuntimeError: Not enough arguments provided: missing keys` in `dask.persist` with mix of `Future` and `Delayed`
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 42/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 停滞
- 技术栈
- python
调研方向
使用报告的版本变更运行最小示例，并将混合 Future/Delayed 情况与两个正常工作的情况进行比较。跟踪 dask.persist 在 dask/base.py、dask/core.py 和 dask/_task_spec.py 中的执行路径，尤其关注报告的 repack 和 missing-key 路径。完成的标准是混合输入不再引发 RuntimeError，同时现有的仅 Future 和仅 Delayed 情况继续正常工作。
由索引模型根据 Issue 内容生成。
描述
Describe the issue:
I get a RuntimeError with some dask-ml code that worked with dask / distributed 2024.10.0 and earlier. With 2024.11.0 and newer, it fails:
Minimal Complete Verifiable Example:
import dask
import dask.distributed
@dask.delayed
def f(x):
return x
def main():
with dask.distributed.Client() as client:
print(dask.persist(client.submit(lambda x: x, 1), f(0)))
if __name__ == "__main__":
main()
On the older versions of dask, that prints out
(<Future: pending, key: lambda-80efb60e0b886931a35d757d5c4b3ac2>, Delayed('f-8a35b8af-26b5-427e-86c8-98b6d8b3f49f'))
Newer versions fail with
Traceback (most recent call last):
File "/Users/tom/gh/dask/dask-ml/bug.py", line 15, in <module>
main()
File "/Users/tom/gh/dask/dask-ml/bug.py", line 12, in main
print(dask.persist(client.submit(lambda x: x, 1), f(0)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/base.py", line 988, in persist
return repack(results)
^^^^^^^^^^^^^^^
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/base.py", line 535, in repack
return simple_get(dsk, out)
^^^^^^^^^^^^^^^^^^^^
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/core.py", line 102, in get
result = execute_graph(dsk2, cache, keys=set(flatten([out])))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/_task_spec.py", line 786, in execute_graph
cache[key] = node(cache)
^^^^^^^^^^^
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/_task_spec.py", line 670, in __call__
self._verify_values(values)
File "/Users/tom/gh/dask/dask-ml/.direnv/python-3.11/lib/python3.11/site-packages/dask/_task_spec.py", line 428, in _verify_values
raise RuntimeError(f"Not enough arguments provided: missing keys {missing}")
RuntimeError: Not enough arguments provided: missing keys {'lambda-80efb60e0b886931a35d757d5c4b3ac2'}
Anything else we need to know?:
It seems to be the combination of Futures and Delayed objects that causes issues. Both of these seem to work fine.
print(dask.persist(client.submit(lambda x: x, 1), client.submit(lambda x: x, 2)) # OK
print(dask.persist(f(0), f(1))) # OK
I'll look for fixes / workarounds once I get up to speed on what's going on.
Environment:
- Dask version:
- Python version:
- Operating System:
- Install method (conda, pip, source):
- 主要语言
- Python
- 星标
- 1.7k
- 派生
- 778
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
dask/distributed 的其他 Issue
-
needs triage
难度 2/5 1-3 小时 新手友好度 72/100
dask/distributed#9366 ·
-
needs triage
难度 2/5 1-3 小时 新手友好度 84/100
dask/distributed#9353 ·
-
documentation
难度 1/5 1-3 小时 新手友好度 82/100
dask/distributed#8304 ·
-
难度 2/5 1-3 小时 新手友好度 74/100
dask/distributed#4816 · 2 条评论 ·
-
documentation good first issue
难度 2/5 1-3 小时 新手友好度 74/100
dask/distributed#2378 · 2 条评论 ·
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
spec-kitty/spec-kitty#5319 ·
维护者通常 1 天内回复
-
backend::vllm diffusion multimodal
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
openai/openai-agents-python#5229 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复