I am trying to extract the labels from a file path of the form:
'/content/UTKFace/26_0_3_20170119181310597.jpg.chip.jpg'
The labels are 26, 0 and 3 in the file name
first I create a list dataset:
data_dir = '/content/UTKFace' data_dir = pathlib.Path(data_dir) list_ds = tf.data.Dataset.list_files(str(data_dir/'*'))
then I define a function that reads the image and gets the labels and use .map() on list_ds
def decode(filename): bits = tf.io.read_file(filename) image = tf.io.decode_jpeg(bits, channels=3) image = tf.image.resize(image, [80, 80]) image = (image - 127.5) / 127.5 parts1 = tf.strings.split(filename, sep='/')[-1] parts2 = tf.strings.split(parts1, sep='_')[0:3] labels = tf.strings.to_number(parts2, tf.int64) return image, labels ds = list_ds.map(decode)
when I print some one of the labels as a sanity check I get this (1):
for i, labels in ds.take(1): print(labels) tf.Tensor([1 0 2], shape=(3,), dtype=int64)
but when I apply .batch() on ds and then try to print all the labels from the dataset, most of the labels are printed but then this error shows up:
--------------------------------------------------------------------------- InvalidArgumentError Traceback (most recent call last) <ipython-input-254-3b19dc87cdce> in <module>() ----> 1 ss = tf.strings.to_number(spl, tf.int64) 4 frames /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs) 199 """Call target, and fall back on dispatchers if there is a TypeError.""" 200 try: --> 201 return target(*args, **kwargs) 202 except (TypeError, ValueError): 203 # Note: convert_to_eager_tensor currently raises a ValueError, not a /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/string_ops.py in string_to_number(input, out_type, name) 477 A `Tensor` of type `out_type`. 478 """ --> 479 return gen_parsing_ops.string_to_number(input, out_type, name) 480 481 /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py in string_to_number(string_tensor, out_type, name) 2311 return _result 2312 except _core._NotOkStatusException as e: -> 2313 _ops.raise_from_not_ok_status(e, name) 2314 except _core._FallbackException: 2315 pass /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name) 6860 message = e.message + (" name: " + name if name is not None else "") 6861 # pylint: disable=protected-access -> 6862 six.raise_from(core._status_to_exception(e.code, message), None) 6863 # pylint: enable=protected-access 6864 /usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value) InvalidArgumentError: StringToNumberOp could not correctly convert string: [Op:StringToNumber]
any ideas what might be causing this?
also regarding (1), I was expecting the output tensor to be tf.Tensor([1, 0, 2], shape=(3,), dtype=int64)
and not tf.Tensor([1 0 2], shape=(3,), dtype=int64)
.
what kind of tensor is this?
Advertisement
Answer
It turns out the error was caused by a few files that were missing the third label from their filename and the tf.strings.to_number() method was trying to convert a substring of the form ‘20170119181310597.jpg.chip.jpg’ to a number.