apache/beam

Python SDK should stage only up-to-date versions of pipeline dependencies defined by requirements file.

开放

#19,968 创建于 2022年6月4日

 (0 条评论) (0 个反应) (0 位负责人)Java (4,097 个派生)batch import
P3bugcoregood first issuepython

仓库指标

星标
 (7,313 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

When staging pipeline dependencies specified in requirements.txt file and transitive deps, we stage entire content of requirements-cache directory on user's machine, which may include dependencies that were cached a while ago and are no longer needed. See [1] for discussion and some ideas how we can avoid staging packages we don't need.

[1] https://lists.apache.org/thread.html/f99820489026a46160505ea9a4c8c4a0eabe8649143c1ee22b7088aa%40%3Cdev.beam.apache.org%3E

Imported from Jira BEAM-8900. Original Jira may contain additional context. Reported by: tvalentyn.

贡献者指南