`ImageFeatureTest`s are failing
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức phù hợp với người mới
- 45/100
Hướng nghiên cứu
Bắt đầu với tensorflow_datasets/core/features/image_feature_test.py và đường dẫn thumbnail đang bị lỗi trong tensorflow_datasets/core/utils/image_utils.py. Chạy các trường hợp ImageFeatureTest mục tiêu với Pillow 12.0.0, sau đó xác minh rằng các trường hợp hình ảnh uint16 được báo cáo đều đạt mà không làm hỏng các bài kiểm tra tính năng hình ảnh hiện có.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Short description
Running the test suite with pillow 12.0.0 gives the following failures:
=========================== short test summary info ============================
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images34 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images35 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images4 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images5 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images10 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images11 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images28 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images29 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
========== 8 failed, 3332 passed, 588 skipped, 773 warnings in 56.61s ==========
Environment information
-
Operating System: NixOS
-
Python version: 3.13.9
-
tensorflow-datasets/tfds-nightlyversion:tensorflow-datasets4.9.9 -
tensorflow/tf-nightlyversion:tensorflow2.20 -
Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ?
-> Not tested
Reproduction instructions
$ pytest
Link to logs
________________________ ImageFeatureTest.test_images34 ________________________
[gw32] linux -- Python 3.13.9 /nix/store/7gl4khq26h625mpqzkiiqsify3hclar1-python3-3.13.9/bin/python3.13
obj = array([[[ 27, 68, 156],
[108, 16, 90],
[ 76, 43, 200],
...,
[189, 72, 101],
... ...,
[159, 21, 92],
[ 48, 35, 23],
[ 40, 150, 248]]], shape=(128, 100, 3), dtype=uint16)
mode = 'I;16'
def fromarray(obj: SupportsArrayInterface, mode: str | None = None) -> Image:
"""
Creates an image memory from an object exporting the array interface
(using the buffer protocol)::
from PIL import Image
import numpy as np
a = np.zeros((5, 5))
im = Image.fromarray(a)
If ``obj`` is not contiguous, then the ``tobytes`` method is called
and :py:func:`~PIL.Image.frombuffer` is used.
In the case of NumPy, be aware that Pillow modes do not always correspond
to NumPy dtypes. Pillow modes only offer 1-bit pixels, 8-bit pixels,
32-bit signed integer pixels, and 32-bit floating point pixels.
Pillow images can also be converted to arrays::
from PIL import Image
import numpy as np
im = Image.open("hopper.jpg")
a = np.asarray(im)
When converting Pillow images to arrays however, only pixel values are
transferred. This means that P and PA mode images will lose their palette.
:param obj: Object with array interface
:param mode: Optional mode to use when reading ``obj``. Since pixel values do not
contain information about palettes or color spaces, this can be used to place
grayscale L mode data within a P mode image, or read RGB data as YCbCr for
example.
See: :ref:`concept-modes` for general information about modes.
:returns: An image object.
.. versionadded:: 1.1.6
"""
arr = obj.__array_interface__
shape = arr["shape"]
ndim = len(shape)
strides = arr.get("strides", None)
try:
typekey = (1, 1) + shape[2:], arr["typestr"]
except KeyError as e:
if mode is not None:
typekey = None
color_modes: list[str] = []
else:
msg = "Cannot handle this data type"
raise TypeError(msg) from e
if typekey is not None:
try:
> typemode, rawmode, color_modes = _fromarray_typemap[typekey]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
E KeyError: ((1, 1, 3), '<u2')
/nix/store/96vld9s60486ifj2pq1v3gdbq19gx145-python3.13-pillow-12.0.0/lib/python3.13/site-packages/PIL/Image.py:3285: KeyError
The above exception was the direct cause of the following exception:
self = <tensorflow_datasets.core.features.image_feature_test.ImageFeatureTest testMethod=test_images34>
make_lib_fail = <function make_pil_fail at 0x7ffb8c17c400>
dtypes = (tf.uint16, <class 'numpy.uint16'>), channels = 3
@parameterized.product(
make_lib_fail=[make_none_fail, make_cv2_fail, make_pil_fail],
dtypes=[
(np.uint8, np.uint8),
(np.uint16, np.uint16),
(tf.uint8, np.uint8),
(tf.uint16, np.uint16),
],
channels=[1, 3, 4],
)
def test_images(self, make_lib_fail, dtypes, channels):
dtype, np_dtype = dtypes
with make_lib_fail() as failing_lib:
if _unsupported_images_for_pil(np_dtype, failing_lib):
return
img = randint(256, size=(128, 100, channels), dtype=np_dtype)
img_other_shape = randint(256, size=(64, 200, channels), dtype=np_dtype)
filename = {
np.uint8: {
1: '6pixels_grayscale.png',
3: '6pixels.png',
4: '6pixels_4chan.png',
},
np.uint16: {
1: '6pixels_grayscale_16bit.png',
3: '6pixels_16bit.png',
4: '6pixels_16bit_4chan.png',
},
}[np_dtype][channels]
img_file_path = os.path.join(
os.path.dirname(__file__), '../../testing/test_data', filename
)
with tf.io.gfile.GFile(img_file_path, 'rb') as f:
img_byte_content = f.read()
img_file_expected_content = np.array(
[
[[0, 255, 0, 255], [255, 0, 0, 255], [255, 0, 255, 255]],
[[0, 0, 255, 255], [255, 255, 0, 255], [126, 127, 128, 255]],
],
dtype=np_dtype,
)[
:, :, :channels
] # Truncate (h, w, 4) -> (h, w, c)
if dtype == np.uint16 or dtype == tf.uint16:
img_file_expected_content *= 257 # Scale int16 images
numpy_array = testing.FeatureExpectationItem(
value=img,
expected=img,
expected_np=img,
)
file_path_as_string = testing.FeatureExpectationItem(
value=img_file_path,
expected=img_file_expected_content,
expected_np=img_file_expected_content,
)
file_path_as_path = testing.FeatureExpectationItem(
value=pathlib.Path(img_file_path),
expected=img_file_expected_content,
expected_np=img_file_expected_content,
)
images_bytes = testing.FeatureExpectationItem(
value=img_byte_content,
expected=img_file_expected_content,
expected_np=img_file_expected_content,
)
img_shape_can_be_dynamic = testing.FeatureExpectationItem(
value=img_other_shape,
expected=img_other_shape,
expected_np=img_other_shape,
)
invalid_type = testing.FeatureExpectationItem(
value=randint(256, size=(128, 128, channels), dtype=np.uint32),
raise_cls=ValueError,
raise_cls_np=ValueError,
raise_msg='dtype should be',
)
tests = [
numpy_array,
file_path_as_string,
file_path_as_path,
images_bytes,
img_shape_can_be_dynamic,
invalid_type,
]
# PIL doesn't support 16-bit images.
if (failing_lib != LibWithImportError.PIL) and np_dtype != np.uint16:
tests.append(
testing.FeatureExpectationItem(
value=PIL.Image.open(img_file_path),
expected=img_file_expected_content,
expected_np=img_file_expected_content,
)
)
> self.assertFeature(
feature=features_lib.Image(shape=(None, None, channels), dtype=dtype),
shape=(None, None, channels),
dtype=dtype,
tests=tests,
test_attributes=dict(
_encoding_format=None,
_use_colormap=False,
),
)
tensorflow_datasets/core/features/image_feature_test.py:177:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tensorflow_datasets/testing/test_utils.py:421: in decorated
f(self, *args, **kwargs)
tensorflow_datasets/testing/feature_test_case.py:186: in assertFeature
self.assertFeatureEagerOnly(
tensorflow_datasets/testing/feature_test_case.py:236: in assertFeatureEagerOnly
run_tests(serialize_fdict=fdict, deserialize_fdict=fdict, feature=feature)
tensorflow_datasets/testing/feature_test_case.py:318: in _run_tests
self.assertFeatureTest(
tensorflow_datasets/testing/feature_test_case.py:463: in assertFeatureTest
self._test_repr(feature, out_numpy)
tensorflow_datasets/testing/feature_test_case.py:483: in _test_repr
text = f.repr_html(ex)
^^^^^^^^^^^^^^^
tensorflow_datasets/core/features/image_feature.py:398: in repr_html
img = utils.create_thumbnail(ex, use_colormap=self._use_colormap)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow_datasets/core/utils/image_utils.py:190: in create_thumbnail
img = PIL_Image.fromarray(ex, mode=mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = array([[[ 27, 68, 156],
[108, 16, 90],
[ 76, 43, 200],
...,
[189, 72, 101],
... ...,
[159, 21, 92],
[ 48, 35, 23],
[ 40, 150, 248]]], shape=(128, 100, 3), dtype=uint16)
mode = 'I;16'
def fromarray(obj: SupportsArrayInterface, mode: str | None = None) -> Image:
"""
Creates an image memory from an object exporting the array interface
(using the buffer protocol)::
from PIL import Image
import numpy as np
a = np.zeros((5, 5))
im = Image.fromarray(a)
If ``obj`` is not contiguous, then the ``tobytes`` method is called
and :py:func:`~PIL.Image.frombuffer` is used.
In the case of NumPy, be aware that Pillow modes do not always correspond
to NumPy dtypes. Pillow modes only offer 1-bit pixels, 8-bit pixels,
32-bit signed integer pixels, and 32-bit floating point pixels.
Pillow images can also be converted to arrays::
from PIL import Image
import numpy as np
im = Image.open("hopper.jpg")
a = np.asarray(im)
When converting Pillow images to arrays however, only pixel values are
transferred. This means that P and PA mode images will lose their palette.
:param obj: Object with array interface
:param mode: Optional mode to use when reading ``obj``. Since pixel values do not
contain information about palettes or color spaces, this can be used to place
grayscale L mode data within a P mode image, or read RGB data as YCbCr for
example.
See: :ref:`concept-modes` for general information about modes.
:returns: An image object.
.. versionadded:: 1.1.6
"""
arr = obj.__array_interface__
shape = arr["shape"]
ndim = len(shape)
strides = arr.get("strides", None)
try:
typekey = (1, 1) + shape[2:], arr["typestr"]
except KeyError as e:
if mode is not None:
typekey = None
color_modes: list[str] = []
else:
msg = "Cannot handle this data type"
raise TypeError(msg) from e
if typekey is not None:
try:
typemode, rawmode, color_modes = _fromarray_typemap[typekey]
except KeyError as e:
typekey_shape, typestr = typekey
msg = f"Cannot handle this data type: {typekey_shape}, {typestr}"
> raise TypeError(msg) from e
E TypeError: Cannot handle this data type: (1, 1, 3), <u2
/nix/store/96vld9s60486ifj2pq1v3gdbq19gx145-python3.13-pillow-12.0.0/lib/python3.13/site-packages/PIL/Image.py:3289: TypeError
------------------------------ Captured log call -------------------------------
WARNING absl:feature.py:71 `FeatureConnector.dtype` is deprecated. Please change your code to use NumPy with the field `FeatureConnector.np_dtype` or use TensorFlow with the field `FeatureConnector.tf_dtype`.
Expected behavior
All tests pass
Additional context
I am working as a nixpkgs maintainer.
Relevant PR: https://github.com/NixOS/nixpkgs/pull/463025
- Ngôn ngữ chính
- Python
- Star
- 4.6k
- Fork
- 1.6k
- Merge trung bình
- 5 giờ 47 phút
- Pull request đã merge (30 ngày)
- 2
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của tensorflow/datasets
-
bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
tensorflow/datasets#11212 · 2 bình luận · 1 reaction ·
-
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 76/100
tensorflow/datasets#11233 ·
-
dataset request
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 20/100
tensorflow/datasets#11228 ·
-
QM9 download links are brokenĐang mởbug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 55/100
tensorflow/datasets#11210 · 1 bình luận ·
-
Độ khó 5/5 Hơn một tuần Mức phù hợp với người mới 1/100
tensorflow/datasets#11196 ·
Tất cả issue của tensorflow/datasets
Issue tương tự
-
bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 90/100
Maintainer thường phản hồi trong vòng 1 ngày
-
https://search.utilibre.orgĐang mởinstance instance add
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
searxng/searx-instances#941 · 1 bình luận ·
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 92/100
FluidNumerics/fluid-walk-blocker#89 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 84/100
Maintainer thường phản hồi trong vòng 1 ngày