apache/beam

Trigger example in the cookbook failed to run on Dataflow

Open

#18,767 创建于 2022年6月3日

在 GitHub 查看
 (2 评论) (0 反应) (1 负责人)Java (4,097 fork)batch import
P3bugexamplesgood first issuejava

仓库指标

Star
 (7,313 star)
PR 合并指标
 (平均合并 6天) (30 天内合并 243 个 PR)

描述

The example org.apache.beam.examples.cookbook.TriggerExample failed to run on Dataflow. Big query returns failure code 409 indicating it's trying to write to an non-empty table but WriteDisposition is set to WRITE_EMPTY (the default value). The example fails with this error even if the output table does not exist before the example is run. 

In fact, the four pipelines corresponding to the four trigger types write to the same table. The pipeline that finishes first will create the table and insert data into it. But the remaining ones can only append to that table. Hence, the problem can be avoided by setting WriteDisposition to WRITE_APPEND.

Imported from Jira BEAM-3844. Original Jira may contain additional context. Reported by: Zhaozhi.

贡献者指南