Impossible to submit Google Dataflow job from Windows machine
#18.734 aperta il 3 giu 2022
Metriche repository
- Star
- (7313 star)
- Metriche merge PR
- (Merge medio 6g) (243 PR mergiate in 30 g)
Descrizione
Got exception when trying to submit Dataflow job from java application running on Windows machine.
java.lang.IllegalArgumentException: Location must be local or on Cloud Storage, got C:\Users\myuser\AppData\Local\Temp\dataflow-job341260414234234.json. Here:
following check is performed: fileLocation.startsWith("/") || fileLocation.startsWith("gs://"),
which prohibits usage of any Windows-style file names.
Workaround is to provide explicit file location on GCS:
dataflowPipelineOptions.setDataflowJobFile("gs://mybucket/dataflowjob.pb");
Imported from Jira BEAM-4380. Original Jira may contain additional context. Reported by: kardashov.