apache/beam

Python SDK should stage only up-to-date versions of pipeline dependencies defined by requirements file.

オープン

#19,968 opened on 2022/06/04

 (0 件のコメント) (0 件のリアクション) (0 人の担当者)Java (4,097 件のフォーク)batch import
P3bugcoregood first issuepython

Repository metrics

Stars
 (7,313 個のスター)
PR merge metrics
 (PR metrics pending)

説明

When staging pipeline dependencies specified in requirements.txt file and transitive deps, we stage entire content of requirements-cache directory on user's machine, which may include dependencies that were cached a while ago and are no longer needed. See [1] for discussion and some ideas how we can avoid staging packages we don't need.

[1] https://lists.apache.org/thread.html/f99820489026a46160505ea9a4c8c4a0eabe8649143c1ee22b7088aa%40%3Cdev.beam.apache.org%3E

Imported from Jira BEAM-8900. Original Jira may contain additional context. Reported by: tvalentyn.

コントリビューターガイド