Open-EO/openeo-geotrellis-extensions

Issues with PNGs

Ouverte

#24 ouverte le 5 juil. 2021

 (5 commentaires) (0 réaction) (1 personne assignée)Scala (5 forks)auto 404
good first issue

Métriques du dépôt

Stars
 (7 étoiles)
Métriques de merge PR
 (Métriques PR en attente)

Description

Copied from: https://github.com/Open-EO/openeo-python-client/issues/219

  1. The files exported by a batch job are returned from the server in the headers with the wrong content type. I got application/octet-stream instead of image/png.
  2. The file doesn't have an extension, the header is content-disposition: inline; filename=out instead of content-disposition: inline; filename=out.png
  3. It seems I can't export a 3 band PNG. I have a (x,y,bands) data cube with the band labels were vv, vh, diff and got an error:
error processing batch job Traceback (most recent call last): File "batch_job.py", line 287, in main run_driver() File "batch_job.py", line 273, in run_driver api_version=api_version, job_dir=job_dir, dependencies=dependencies, user_id=user_id File "/data1/hadoop/yarn/local/usercache/openeo/appcache/application_1624287832965_16431/container_e4942_1624287832965_16431_01_000002/venv/lib64/python3.6/site-packages/openeogeotrellis/utils.py", line 37, in memory_logging_wrapper return function(*args, **kwargs) File "batch_job.py", line 347, in run_job assets_metadata = result.write_assets(str(output_file)) File "/data1/hadoop/yarn/local/usercache/openeo/appcache/application_1624287832965_16431/container_e4942_1624287832965_16431_01_000002/venv/lib64/python3.6/site-packages/openeo_driver/save_result.py", line 79, in write_assets return self.cube.write_assets(filename=directory, format=self.format, format_options=self.options) File "/data1/hadoop/yarn/local/usercache/openeo/appcache/application_1624287832965_16431/container_e4942_1624287832965_16431_01_000002/venv/lib64/python3.6/site-packages/openeogeotrellis/geopysparkdatacube.py", line 1209, in write_assets self._get_jvm().org.openeo.geotrellis.png.package.saveStitched(spatial_rdd.srdd.rdd(), filename, crop_extent) File "/usr/hdp/3.1.4.0-315/spark2/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__ answer, self.gateway_client, self.target_id, self.name) File "/usr/hdp/3.1.4.0-315/spark2/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value format(target_id, ".", name), value) py4j.protocol.Py4JJavaError: An error occurred while calling z:org.openeo.geotrellis.png.package.saveStitched. : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 18.0 failed 4 times, most recent failure: Lost task 0.3 in stage 18.0 (TID 50, epod057.vgt.vito.be, executor 4): java.lang.ClassCastException: java.lang.Integer cannot be cast to geotrellis.layer.SpatialKey at geotrellis.spark.partition.PartitionerIndex$SpatialPartitioner$.toIndex(PartitionerIndex.scala:37) at geotrellis.spark.partition.SpacePartitioner.getPartition(SpacePartitioner.scala:45) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:151) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:109) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1651) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1639) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1638) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1638) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831) at scala.Option.foreach(Option.scala:257) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1872) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1821) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1810) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2039) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2060) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2079) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2104) at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:945) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) at org.apache.spark.rdd.RDD.collect(RDD.scala:944) at org.apache.spark.RangePartitioner$.sketch(Partitioner.scala:309) at org.apache.spark.RangePartitioner.<init>(Partitioner.scala:171) at org.apache.spark.RangePartitioner.<init>(Partitioner.scala:151) at org.apache.spark.rdd.OrderedRDDFunctions$$anonfun$sortByKey$1.apply(OrderedRDDFunctions.scala:62) at org.apache.spark.rdd.OrderedRDDFunctions$$anonfun$sortByKey$1.apply(OrderedRDDFunctions.scala:61) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) at org.apache.spark.rdd.OrderedRDDFunctions.sortByKey(OrderedRDDFunctions.scala:61) at org.openeo.geotrellis.png.package$.saveStitched(package.scala:21) at org.openeo.geotrellis.png.package.saveStitched(package.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to geotrellis.layer.SpatialKey at geotrellis.spark.partition.PartitionerIndex$SpatialPartitioner$.toIndex(PartitionerIndex.scala:37) at geotrellis.spark.partition.SpacePartitioner.getPartition(SpacePartitioner.scala:45) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:151) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:109) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more

Guide contributeur