[Failing Test]: BigqueryClient.queryUnflattened fails BigQuery ITs on a transient 503
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 77/100
Research direction
Start in BigqueryClient.queryUnflattened at BigqueryClient.java:345, then compare its polling and cleanup behavior with the retrying methods named in the issue, especially queryWithRetries. Check the BigQuery integration tests that call this method, including StorageApiDataTriggeredSchemaUpdateIT.verifyDataWritten; done means transient 503 responses are retried and the temporary dataset is cleaned up when the query returns no rows or throws.
Written by the indexing model from the issue text.
Description
beam_PreCommit_Java_GCP_IO_Direct_BigQuery failed on #40790 in StorageApiDataTriggeredSchemaUpdateIT.testDataTriggeredSchemaUpgradeExactlyOnce:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 503 Service Unavailable
GET https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/queries/job_...
"reason": "backendError"
"message": "Error encountered during execution. Retrying may solve the problem."
at org.apache.beam.sdk.io.gcp.testing.BigqueryClient.queryUnflattened(BigqueryClient.java:345)
at org.apache.beam.sdk.io.gcp.bigquery.StorageApiDataTriggeredSchemaUpdateIT.verifyDataWritten(StorageApiDataTriggeredSchemaUpdateIT.java:367)
The pipeline had finished, and the test failed while checking its output. BigqueryClient.queryUnflattened polls jobs.getQueryResults with no retry, so a single 503 fails the test. The other BigqueryClient methods, such as createNewDataset, createNewTable and queryWithRetries, already retry with backoff. 12 test files call queryUnflattened.
queryUnflattened also leaves its temporary dataset behind when the query returns no rows or throws.
Failed job: https://github.com/apache/beam/actions/runs/40205636577/job/121205844102
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 290
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/beam
-
awaiting triage bug P2 python
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
[Feature Request]: Support Protobuf 7.x in Python SDKPossibly taken @sofiathoren claimed this 12 days ago. OpenP2 python
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/beam#39624 · 2 reactions ·
Maintainers usually reply within 1 day
-
[Bug]: PubsubIO used in batch incorrect batch cutoff sizePossibly taken @1fanwang claimed this 62 days ago. Openbug io P3 pinned pubsub
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/beam#28011 · 4 comments ·
Maintainers usually reply within 1 day
-
build P3 sub-task
Difficulty 1/5 Under an hour Newbie friendliness 68/100
Maintainers usually reply within 1 day
Similar issues
-
Bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
protobuf
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
FasterXML/jackson-dataformats-binary#874 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
Add the AI-disclosure footer to README.mdPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Add the AI-disclosure footer to README.mdPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
HexaHexes/infinite-dungeon-spawn-safety#17 ·
Maintainers usually reply within 1 day