P2gcpgood first issueionew featurepython
Repository metrics
- Stars
- (7,313 stars)
- PR merge metrics
- (平均マージ 6d) (30d で 243 merged PRs)
説明
What would you like to happen?
There can be a collision when 2 pipelines using templates are loading to BigQuery at the same time to the same temp_location.
@baeminbo found there is code to use unique_id [1], but it seems that templates can re-use the same uuid. It is fixed for Java[2] by moving the UUID generation into a DoFn of a ParDo
[1] https://github.com/apache/beam/blob/v2.34.0/sdks/python/apache_beam/io/gcp/bigquery.py#L2399 [2] https://github.com/apache/beam/blob/v2.34.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1094-L1105
Issue Priority
Priority: 2
Issue Component
Component: io-py-gcp