InvalidArgumentError: StringToNumberOp could not correctly convert string

I am trying to extract the labels from a file path of the form:

'/content/UTKFace/26_0_3_20170119181310597.jpg.chip.jpg'

The labels are 26, 0 and 3 in the file name

first I create a list dataset:

data_dir = '/content/UTKFace'

data_dir = pathlib.Path(data_dir)

list_ds = tf.data.Dataset.list_files(str(data_dir/'*'))

then I define a function that reads the image and gets the labels and use .map() on list_ds

def decode(filename):
  bits = tf.io.read_file(filename)
  image = tf.io.decode_jpeg(bits, channels=3)
  image = tf.image.resize(image, [80, 80])
  image = (image - 127.5) / 127.5
  parts1 = tf.strings.split(filename, sep='/')[-1]
  parts2 = tf.strings.split(parts1, sep='_')[0:3]
  labels = tf.strings.to_number(parts2, tf.int64)
  return image, labels

ds = list_ds.map(decode)

when I print some one of the labels as a sanity check I get this (1):

for i, labels in ds.take(1):
  print(labels)

tf.Tensor([1 0 2], shape=(3,), dtype=int64)

but when I apply .batch() on ds and then try to print all the labels from the dataset, most of the labels are printed but then this error shows up:

---------------------------------------------------------------------------

InvalidArgumentError                      Traceback (most recent call last)

<ipython-input-254-3b19dc87cdce> in <module>()
----> 1 ss = tf.strings.to_number(spl, tf.int64)

4 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
    199     """Call target, and fall back on dispatchers if there is a TypeError."""
    200     try:
--> 201       return target(*args, **kwargs)
    202     except (TypeError, ValueError):
    203       # Note: convert_to_eager_tensor currently raises a ValueError, not a

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/string_ops.py in string_to_number(input, out_type, name)
    477     A `Tensor` of type `out_type`.
    478   """
--> 479   return gen_parsing_ops.string_to_number(input, out_type, name)
    480 
    481 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_parsing_ops.py in string_to_number(string_tensor, out_type, name)
   2311       return _result
   2312     except _core._NotOkStatusException as e:
-> 2313       _ops.raise_from_not_ok_status(e, name)
   2314     except _core._FallbackException:
   2315       pass

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6860   message = e.message + (" name: " + name if name is not None else "")
   6861   # pylint: disable=protected-access
-> 6862   six.raise_from(core._status_to_exception(e.code, message), None)
   6863   # pylint: enable=protected-access
   6864 

/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: StringToNumberOp could not correctly convert string:  [Op:StringToNumber]

any ideas what might be causing this?

also regarding (1), I was expecting the output tensor to be tf.Tensor([1, 0, 2], shape=(3,), dtype=int64) and not tf.Tensor([1 0 2], shape=(3,), dtype=int64). what kind of tensor is this?

Answer

It turns out the error was caused by a few files that were missing the third label from their filename and the tf.strings.to_number() method was trying to convert a substring of the form ‘20170119181310597.jpg.chip.jpg’ to a number.