java tensorflow and maven version is org.tensorflow:libtensorflow:1.15.0 after session.runner.run() for many times, the memory grow higher and higher then oom
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- java, tensorflow
- Domain
- machine-learning
Research direction
Start by reproducing the reported workload using SavedModelBundle, session.runner(), runner.run(), and the repeated 100-QPS prediction pattern shown in the issue. Trace ownership and closure of the fed input tensors and returned output tensors while monitoring memory; done means the repeated predictions no longer grow memory until OOM.
Written by the indexing model from the issue text.
Description
my test code with language scala ,this is one predict, we will predict 100 QPS for a docker
val config = ConfigProto.newBuilder
.putDeviceCount("CPU", Runtime.getRuntime.availableProcessors)
.setInterOpParallelismThreads(8)
.setIntraOpParallelismThreads(8)
.setOperationTimeoutInMs(3000)
.build
val options = RunOptions.newBuilder
.setTimeoutInMs(5000)
.build
val modelBundle = SavedModelBundle
.loader(s"$path")
.withTags("serve")
.withConfigProto(config.toByteArray)
.withRunOptions(options.toByteArray)
.load
val kernel = modelBundle.session
val data = Map("tensor1" -> Seq(0.1f,0.122f),……)
val runner = kernel.runner()
val inputTensorList: util.ArrayList[Tensor[java.lang.Float]] = new util.ArrayList[Tensor[java.lang.Float]]()
data.map{
case (tensorName, featureId) => {
val dataInput:FloatBuffer = FloatBuffer.allocate(featureId.size)
featureId.foreach(featureValue => {
dataInput.put(featureValue)
})
dataInput.asInstanceOf[Buffer].flip()
val tensorShape:Array[Long] = Array(1,featureId.size)
val tensor = Tensor.create(tensorShape,dataInput)
runner.feed(tensorName,tensor)
inputTensorList.add(tensor)
}
}
for(i <- 0 until 2 ){
runner.fetch("StatefulPartitionedCall",i)
}
val output = runner.run.asScala
val scores:Array[Float] = output.map(ten => {
val tensorData: Array[Array[Float]] = ten.copyTo(Array.ofDim[Float](ten.shape()(0).toInt, ten.shape()(1).toInt))
tensorData(0).head
}).toArray
inputTensorList.asScala.foreach(_.close())
output.foreach(_.close())
- Dominant language
- Java
- Stars
- 928
- Forks
- 227
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from tensorflow/java
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
tensorflow/java#653 · 1 comment · 4 reactions ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
tensorflow/java#621 · 4 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
tensorflow/java#617 · 3 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
tensorflow/java#615 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
tensorflow/java#614 · 1 comment ·
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
elastic/gradle-plugins#157 ·
-
enhancement Tools
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
apache/rocketmq-dashboard#5008 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
DETECT_PARAMETER_NAMES=false silently disables @ConstructorProperties-based Creator detection too Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
FasterXML/jackson-databind#6229 ·