I am trying to extract the labels from a file path of the form:
'/content/UTKFace/26_0_3_20170119181310597.jpg.chip.jpg'
The labels are 26, 0 and 3 in the file name
first I create a list dataset:
data_dir = '/content/UTKFace'
data_dir = pathlib.Path(data_dir)
list_ds = tf.data.Dataset.list_files(str(data_dir/'*'))
then I define a function that reads the image and gets the labels and use .map() on list_ds
def decode(filename):
bits = tf.io.read_file(filename)
image = tf.io.decode_jpeg(bits, channels=3)
image = tf.image.resize(image, [80, 80])
image = (image - 127.5) / 127.5
parts1 = tf.strings.split(filename, sep='/')[-1]
parts2 = tf.strings.split(parts1, sep='_')[0:3]
labels = tf.strings.to_number(parts2, tf.int64)
return image, labels
ds = list_ds.map(decode)
when I print some one of the labels as a sanity check I get this (1):
for i, labels in ds.take(1):
print(labels)
tf.Tensor([1 0 2], shape=(3,), dtype=int64)
but when I apply .batch() on ds and then try to print all the labels from the dataset, most of the labels are printed but then this error shows up:
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-254-3b19dc87cdce> in <module>()
----> 1 ss = tf.strings.to_number(spl, tf.int64)
4 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 """Call target, and fall back on dispatchers if there is a TypeError."""
200 try:
--> 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/string_ops.py in string_to_number(input, out_type, name)
477 A `Tensor` of type `out_type`.
478 """
--> 479 return gen_parsing_ops.string_to_number(input, out_type, name)
480
481
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py in string_to_number(string_tensor, out_type, name)
2311 return _result
2312 except _core._NotOkStatusException as e:
-> 2313 _ops.raise_from_not_ok_status(e, name)
2314 except _core._FallbackException:
2315 pass
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
6860 message = e.message + (" name: " + name if name is not None else "")
6861 # pylint: disable=protected-access
-> 6862 six.raise_from(core._status_to_exception(e.code, message), None)
6863 # pylint: enable=protected-access
6864
/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)
InvalidArgumentError: StringToNumberOp could not correctly convert string: [Op:StringToNumber]
any ideas what might be causing this?
also regarding (1), I was expecting the output tensor to be tf.Tensor([1, 0, 2], shape=(3,), dtype=int64)
and not tf.Tensor([1 0 2], shape=(3,), dtype=int64)
.
what kind of tensor is this?
Advertisement
Answer
It turns out the error was caused by a few files that were missing the third label from their filename and the tf.strings.to_number() method was trying to convert a substring of the form ‘20170119181310597.jpg.chip.jpg’ to a number.