How to use TfRecordDataset DatasetToTfRecord tf.io.tfRecordReader

Open
#452 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Stale
Tech stack
java, tensorflow
Domain
data, machine-learning

Research direction

Start with the TfRecordDataset, DatasetToTfRecord, and tf.io.tfRecordReader entry points shown in the issue, and compare their Java APIs with the requested Python usage. Done means providing a working tensorflow-java example that reads TFRecord data and explains how these APIs are used.

Written by the indexing model from the issue text.

Description

tensorflow-java 0.4
spark 3.1
java 11

Hi :
Now I use tensorflow-java to read tfrecord file ,but can not get the data, and our not have example for it ,the TfRecordDataset DatasetToTfRecord tf.io.tfRecordReader java class have not same api like python ,could we give some example for how to use them. thank


    import org.tensorflow.{Operand, Session,EagerSession}
    import org.tensorflow.op.Ops
    import org.tensorflow.op.data.TfRecordDataset
    import org.tensorflow.op.data.{DatasetToTfRecord, TfRecordDataset}
    val session = EagerSession.create
    val tf = Ops.create(session)
    val  scope = tf.scope()
//    val fileName  =tf.constant( "/Users/zhanghaining/Downloads/tfrecord-kk2-test/")
    val fileName = tf.constant("/Users/zhanghaining/Downloads/BigDL/spark/dl/src/test/resources/tf/mnist_train.tfrecord")
    val compress = tf.constant("")
    val bufferSize = tf.constant(0l)
    val recordDataSet = TfRecordDataset.create(scope,fileName,compress,bufferSize)

    val record = DatasetToTfRecord.create(scope, recordDataSet,fileName,compress)

    val reader =  tf.io.tfRecordReader()

    println(record.op().name() )
    println(record.op().`type`())
    println(recordDataSet.op().numOutputs() )
    println(recordDataSet.asOutput().dataType())
Dominant language
Java
Stars
928
Forks
227
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tensorflow/java

All issues in tensorflow/java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.