Raise a clear error for values that cannot be encoded as an image - #8450
Open
LeSingh1 wants to merge 1 commit into
Open
Raise a clear error for values that cannot be encoded as an image#8450LeSingh1 wants to merge 1 commit into
LeSingh1 wants to merge 1 commit into
Conversation
Putting an unsupported value in an Image column fails with an error from deep
inside the encoder rather than a message about the image:
Dataset.from_dict({"im": [123]}, features=Features({"im": Image()}))
AttributeError: 'int' object has no attribute 'get'
Image.encode_example handles str, Path, bytes, ndarray and PIL images, then
assumes anything else is a mapping and calls value.get("path") on it. The Video
feature already guards this exact case with
'TypeError: Unsupported encode_example type: ...', so reuse that message.
A 0-dimensional array holds no image either and raised
'IndexError: tuple index out of range' from encode_np_array, so reject it with
an explicit message too.
Mappings that are missing 'path' and 'bytes' keep their existing ValueError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Putting an unsupported value in an
Image()column fails with an error from deep inside the encoder rather than a message about the image:Same for a float, a bool, a set, or any other unsupported object. A 0-dimensional numpy array fails differently but just as opaquely:
Cause
Image.encode_examplehandlesstr,Path,bytes,np.ndarrayandPIL.Image.Image, and then falls straight through to the mapping branches:so any value that is not one of the supported types and not a mapping raises
AttributeErroron.get.The
Videofeature already guards exactly this case:Imagesimply never got the same guard.Fix
TypeError: Unsupported encode_example type: ...messageVideoalready uses, so the two features report this identically;encode_np_arraywith an explicitValueErrorinstead of letting the shape indexing blow up.Mappingis used rather thandictso that any mapping type keeps working. Mappings that are missing bothpathandbytesstill raise the existing, already explicitValueError, and every currently supported input is unchanged —str,Path,bytes, dicts, PIL images and 1D/2D/3D arrays all still encode exactly as before.Tests
test_image_feature_encode_example_unsupported_typecoversint,float,bool,setand a plain object;test_image_feature_encode_example_zero_dimensional_arraycovers the 0-d array through bothImage.encode_exampleandencode_np_array;test_image_feature_encode_example_missing_keyspins the existingValueErrorfor a mapping without usable keys, so this change cannot swallow it.The first two fail on
main(6 parametrizations); the third passes before and after and is there as a regression guard.tests/features/,tests/test_formatting.py,tests/test_table.pyandtests/test_arrow_dataset.py: 1068 passed, 0 failures.