How to load a dataset from google cloud storage with tensorflow cloud?

Open
#136 2 comments 0 reactions 1 assignee View on GitHub

@pavithrasv is already working on this.

Since Aug 20, 2020.

Assessment

This issue has not been assessed yet.

Description

bug

Tensorflow cloud configuration:

  GCP_BUCKET = "stereo-train"

  tfc.run(
    requirements_txt="requirements.txt",
    chief_config=tfc.MachineConfig(
      cpu_cores=8,
      memory=30,
      accelerator_type=tfc.AcceleratorType.NVIDIA_TESLA_T4,
      accelerator_count=1,
    ),
    docker_image_bucket_name=GCP_BUCKET,
  )

And I have a bucket called gs://stereo-train that contains the dataset. The exact location of the dataset is:

gs://stereo-train/data_scene_flow/training/dat

However, when using this location like so:

tf.keras.preprocessing.image_dataset_from_directory("gs://stereo-train/data_scene_flow/training/dat", image_size=(375,1242),\
                                                         batch_size=6, shuffle=False, label_mode=None)

Behavior:

getting the error that "gs://stereo-train/data_scene_flow/training/dat" doesn't exist

Expected behavior:
tf.keras.preprocessing.image_dataset_from_directory should know that there's a gs bucket associated with the account and the dataset should be loaded.

Dominant language
Python
Stars
383
Forks
93
Avg merge
1d 3h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tensorflow/cloud

All issues in tensorflow/cloud

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.