apache/beam

[Task]: Update BigQueryIO.setTriggeringFrequency documentation

Open

#23,949 建立於 2022年11月2日

在 GitHub 查看
 (20 留言) (0 反應) (3 負責人)Java (7,313 star) (4,097 fork)batch import
P2bigquerygcpgood first issueiojavatask

描述

What needs to happen?

BigQueryIO currently documents triggering frequency as:

This is only applicable when the write method is set to {@link Method#FILE_LOADS}, and only when writing an unbounded {@link PCollection}.

However, we use triggering frequency for STORAGE_WRITE_API and STORAGE_API_AT_LEAST_ONCE as well. This documentation should be updated to reflect this.

In addition, during the validation component of BigQueryIO.write().expand(), we check for unbounded && (file loads || storage write api) to see if a triggering frequency is set. This should probably also be checked if storage write at least once is set.

Issue Priority

Priority: 2

Issue Component

Component: io-java-gcp

貢獻者指南

技術棧
java
領域
documentationbackenddata
議題類型
documentation
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
half day
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
blocked
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
JavaApache Beam SDKBigQueryIO basics
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
60
研究方向
Investigate the BigQueryIO source code, specifically the setTriggeringFrequency method documentation and the validation logic in BigQueryIO.write().expand(). The issue comments may indicate further context. Update the Javadoc to mention STORAGE WRITE API and STORAGE API AT LEAST ONCE, and potentially add a validation check for those methods.