tfless_tfds requires TensorFlow
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 42/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Estancado
- Stack tecnológico
- python, tensorflow
- Área
- documentation
Línea de trabajo
Comienza con la página de documentación de tfless_tfds y ejecuta su ejemplo de Python sin TensorFlow, comparando el resultado con el ExtractError informado en tensorflow_datasets/core/download/extractor.py. Se considera terminado cuando el requisito de TensorFlow y el comportamiento esperado del ejemplo coincidan con lo que realmente hace la implementación.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Short description
Code in documentation to demonstrate tfless TFDS does actually require tensorflow to run.
Environment information
-
Operating System: Windows 11
-
Python version: 3.12
-
tensorflow-datasets/tfds-nightlyversion: tfds-nightly==4.9.9.dev202505300044 -
tensorflow/tf-nightlyversion: Not installed -
Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ? Yes
Reproduction instructions
Run the code from this page in the documentation: https://www.tensorflow.org/datasets/tfless_tfds#:~:text=%25%25writefile%20no_tensorflow,be%20decoded.%27)
import os
os.environ.pop("TFDS_DATA_DIR", None)
import tensorflow_datasets as tfds
try:
import tensorflow as tf
except ImportError:
print("No TensorFlow found...")
ds = tfds.data_source(
"fashion_mnist",
try_gcs=False,
)
print("...but the data source could still be loaded...")
ds["train"][0]
print("...and the records can be decoded.")
Link to logs
If applicable, <link to gist with logs, stack trace>
PS C:\Users\username\Code\myproject> & c:/Users/username/Code/myproject/build_venv/Scripts/python.exe c:/Users/username/Code/myproject/tryout.py
No TensorFlow found...
WARNING:absl:Variant folder C:\Users\username\tensorflow_datasets\fashion_mnist\3.0.1 has no dataset_info.json
Downloading and preparing dataset Unknown size (download: Unknown size, generated: Unknown size, total: Unknown size) to C:\Users\username\tensorflow_datasets\fashion_mnist\3.0.1...
Dl Completed...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 47.35 url/s]
Dl Size...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 4422102/4422102 [00:00<00:00, 209400396.35 MiB/s]
***************************************************************
Failed to import TensorFlow. Please note that TensorFlow is not installed by default when you install TFDS. This allows you to choose to install either `tf-nightly` or `tensorflow`. Please install the most recent version of TensorFlow, by following instructions at https://tensorflow.org/install.
***************************************************************
Dl Completed...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 64.53 url/s]
Dl Size...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 4427250/4427250 [00:00<00:00, 142845743.17 MiB/s]
***************************************************************
Failed to import TensorFlow. Please note that TensorFlow is not installed by default when you install TFDS. This allows you to choose to install either `tf-nightly` or `tensorflow`. Please install the most recent version of TensorFlow, by following instructions at https://tensorflow.org/install.
***************************************************************
Dl Completed...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 85.67 url/s]
Dl Size...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 30849130/30849130 [00:00<00:00, 880939482.81 MiB/s]
Dl Completed...: 75%|███████████████████████████████████████████████████████████████████████████████████████▊ | 3/4 [00:00<00:00, 81.10 url/s]
Failed to import TensorFlow. Please note that TensorFlow is not installed by default when you install TFDS. This allows you to choose to install either `tf-nightly` or `tensorflow`. Please install the most recent version of TensorFlow, by following instructions at https://tensorflow.org/install.
***************************************************************
Extraction completed...: 0 file [00:00, ? file/s]█████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 102.53 url/s]
Dl Size...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 30878645/30878645 [00:00<00:00, 741591032.26 MiB/s]
Dl Completed...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 96.07 url/s]
***************************************************************
Failed to import TensorFlow. Please note that TensorFlow is not installed by default when you install TFDS. This allows you to choose to install either `tf-nightly` or `tensorflow`. Please install the most recent version of TensorFlow, by following instructions at https://tensorflow.org/install.
Traceback (most recent call last):
***************************************************************
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\extractor.py", line 106, in _extract
for path, handle in iter_archive(from_path, method):
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\extractor.py", line 220, in iter_gzip
with _open_or_pass(arch_f) as fobj:
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\extractor.py", line 166, in _open_or_pass
with tf.io.gfile.GFile(path_or_fobj, 'rb') as f_obj:
^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\etils\epy\lazy_imports_utils.py", line 118, in __getattr__
return getattr(self._module, name)
^^^^^^^^^^^^
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\functools.py", line 995, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\etils\epy\lazy_imports_utils.py", line 79, in _module
module = importlib.import_module(self.module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1318, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'tensorflow'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\username\Code\myproject\tryout.py", line 12, in <module>
ds = tfds.data_source(
^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\logging\__init__.py", line 176, in __call__
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\load.py", line 836, in data_source
_download_and_prepare_builder(dbuilder, download, download_and_prepare_kwargs)
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\load.py", line 518, in _download_and_prepare_builder
dbuilder.download_and_prepare(**download_and_prepare_kwargs)
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\logging\__init__.py", line 176, in __call__
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 763, in download_and_prepare
self._download_and_prepare(
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 1808, in _download_and_prepare
split_infos = self._generate_splits(dl_manager, download_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 1758, in _generate_splits
split_generators = self._split_generators( # pylint: disable=unexpected-keyword-arg
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\image_classification\mnist.py", line 119, in _split_generators
mnist_files = dl_manager.download_and_extract(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\download_manager.py", line 754, in download_and_extract
return _map_promise(self._download_extract, url_or_urls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\download_manager.py", line 782, in _map_promise
res = tree.map_structure(lambda p: p.get(), all_promises) # Wait promises
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tree\__init__.py", line 428, in map_structure
[func(*args) for args in zip(*map(flatten, structures))])
^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\download_manager.py", line 782, in <lambda>
res = tree.map_structure(lambda p: p.get(), all_promises) # Wait promises
^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\promise\promise.py", line 512, in get
return self._target_settled_value(_raise=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\promise\promise.py", line 516, in _target_settled_value
return self._target()._settled_value(_raise)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\promise\promise.py", line 226, in _settled_value
reraise(type(raise_val), raise_val, self._traceback)
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\six.py", line 724, in reraise
raise value
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\promise\promise.py", line 844, in handle_future_result
resolve(future.result())
^^^^^^^^^^^^^^^
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\_base.py", line 401, in __get_result
raise self._exception
File "C:\Users\username\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\username\Code\myproject\build_venv\Lib\site-packages\tensorflow_datasets\core\download\extractor.py", line 128, in _extract
raise ExtractError(msg) from err
tensorflow_datasets.core.download.extractor.ExtractError: Error while extracting C:\Users\username\tensorflow_datasets\downloads\fashion_mnist\fashion-mnist_t10k-images-idx3-ubyteNG5VuUjZc6l-WNI1Hd4WpIS9QV1FlSl2M7sI8D22oHM.gz to C:\Users\username\tensorflow_datasets\downloads\extracted\GZIP.fashion-mnist_t10k-images-idx3-ubyteNG5VuUjZc6l-WNI1Hd4WpIS9QV1FlSl2M7sI8D22oHM.gz: No module named 'tensorflow'
Expected behavior
I expect the output from the documentation:
No TensorFlow found...
...but the data source could still be loaded...
WARNING:absl:OpenCV is not installed. We recommend using OpenCV because it is faster according to our benchmarks. Defaulting to PIL to decode images...
...and the records can be decoded.
Additional context
The problematic line seems to be here: https://github.com/tensorflow/datasets/blob/3ff4b37b9b4afd8ae7fd8c3d96d309ebca21b4ba/tensorflow_datasets/core/download/extractor.py#L166
Here, tf.io is used.
- Lenguaje dominante
- Python
- Estrellas
- 4.6k
- Forks
- 1.6k
- Merge medio
- 5 h 47 min
- PR fusionados (30 d)
- 2
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de tensorflow/datasets
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
tensorflow/datasets#11212 · 2 comentarios · 1 reacción ·
-
Dificultad 3/5 1-2 días Aptitud para principiantes 76/100
tensorflow/datasets#11233 ·
-
dataset request
Dificultad 4/5 3-5 días Aptitud para principiantes 20/100
tensorflow/datasets#11228 ·
-
QM9 download links are brokenAbiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 55/100
tensorflow/datasets#11210 · 1 comentario ·
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 1/100
tensorflow/datasets#11196 ·
Todos los issues de tensorflow/datasets
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
kornia/kornia#5263 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Metadata correction for W16-5400Abiertoapproved correction metadata
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
acl-org/acl-anthology#10133 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
BasedHardware/omi#20084 ·
Los mantenedores suelen responder en 1 día
-
bug needs-acceptance wg/evaluation-quality
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
vllm-project/semantic-router#4424 ·
Los mantenedores suelen responder en 1 día