Having java.lang.NoSuchMethodError: org.tensorflow.framework.GraphDef.toByteString()

Open
  • Is something wrong?
#179 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
scala, tensorflow
Domain
data-engineering, machine-learning

Research direction

Start with org.tensorframes.impl.TensorFlowOps.scala at graphSerial (line 69) and compare the listed TensorFlow, TensorFrames, Spark Deep Learning, Spark 2.4.5, and Scala 2.11.12 dependencies. Reproduce the DeepImageFeaturizer pipeline fit and trace which dependency provides GraphDef.toByteString(). Done means the pipeline runs without this NoSuchMethodError.

Written by the indexing model from the issue text.

Description

Hi
I want to use DeepImageFeaturizer combined with spark ML Logistic regression in Spark (2.4.5) / scala 2.11.12 but it's not working. I'm trying to resolve it for many days.

I have this issue :
java.lang.NoSuchMethodError: org.tensorflow.framework.GraphDef.toByteString()Lorg/tensorframes/protobuf3shade/ByteString;

It seems a library is missing but i think I've already referenced all the needed ones :

delta-core_2.11-0.6.0.jar
libtensorflow-1.15.0.jar
libtensorflow_jni-1.15.0.jar
libtensorflow_jni_gpu-1.15.0.jar
proto-1.15.0.jar
scala-logging-api_2.11-2.1.2.jar
scala-logging-slf4j_2.11-2.1.2.jar
scala-logging_2.11-3.9.2.jar
spark-deep-learning-1.5.0-spark2.4-s_2.11.jar
spark-sql-kafka-0-10_2.11-2.4.5.jar
spark-tensorflow-connector_2.11-1.6.0.jar
tensorflow-1.15.0.jar
tensorflow-hadoop-1.15.0.jar
tensorframes-0.8.2-s_2.11.jar

Full trace :

20/05/15 21:17:28 DEBUG impl.TensorFlowOps$: Outputs: Set(InceptionV3_sparkdl_output__)
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:65)
	at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
Caused by: java.lang.NoSuchMethodError: org.tensorflow.framework.GraphDef.toByteString()Lorg/tensorframes/protobuf3shade/ByteString;
	at org.tensorframes.impl.TensorFlowOps$.graphSerial(TensorFlowOps.scala:69)
	at org.tensorframes.impl.TensorFlowOps$.analyzeGraphTF(TensorFlowOps.scala:114)
	at org.tensorframes.impl.DebugRowOps.mapRows(DebugRowOps.scala:408)
	at com.databricks.sparkdl.DeepImageFeaturizer.transform(DeepImageFeaturizer.scala:135)
	at org.apache.spark.ml.Pipeline$$anonfun$fit$2.apply(Pipeline.scala:161)
	at org.apache.spark.ml.Pipeline$$anonfun$fit$2.apply(Pipeline.scala:149)
	at scala.collection.Iterator$class.foreach(Iterator.scala:891)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
	at scala.collection.IterableViewLike$Transformed$class.foreach(IterableViewLike.scala:44)
	at scala.collection.SeqViewLike$AbstractTransformed.foreach(SeqViewLike.scala:37)
	at org.apache.spark.ml.Pipeline.fit(Pipeline.scala:149)

Can someone of the team can tell me what is going wrong ?
thanks for your support

Dominant language
Scala
Stars
743
Forks
158
No merged PRs in 30d

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/tensorframes

All issues in databricks/tensorframes

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.