Impossible to submit Google Dataflow job from Windows machine
#18,734 opened on 2022年6月3日
Repository metrics
- Stars
- (7,313 stars)
- PR merge metrics
- (平均マージ 6d) (30d で 243 merged PRs)
説明
Got exception when trying to submit Dataflow job from java application running on Windows machine.
java.lang.IllegalArgumentException: Location must be local or on Cloud Storage, got C:\Users\myuser\AppData\Local\Temp\dataflow-job341260414234234.json. Here:
following check is performed: fileLocation.startsWith("/") || fileLocation.startsWith("gs://"),
which prohibits usage of any Windows-style file names.
Workaround is to provide explicit file location on GCS:
dataflowPipelineOptions.setDataflowJobFile("gs://mybucket/dataflowjob.pb");
Imported from Jira BEAM-4380. Original Jira may contain additional context. Reported by: kardashov.