Keras tuner Bayesian Optmization graph error

I am trying to optimize a convolutional neural network with Bayesian Optimization algorithm provided in keras tuner library.

When I perform the line: tuner_cnn.search(datagen.flow(X_trainRusReshaped,Y_trainRusHot), epochs=50, batch_size=256) I encounter this error: InvalidArgumentError: Graph execution error

One-Hot-Encode y_train and y_test as the following:

y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

X_trainShape = X_train.shape[1]*X_train.shape[2]*X_train.shape[3]
X_testShape = X_test.shape[1]*X_test.shape[2]*X_test.shape[3]
X_trainFlat = X_train.reshape(X_train.shape[0], X_trainShape)
X_testFlat = X_test.reshape(X_test.shape[0], X_testShape)
# One-hot-encoding
Y_trainRusHot = to_categorical(Y_trainRus, num_classes = 2)
Y_testRusHot = to_categorical(Y_testRus, num_classes = 2)

I defined my model builder like that:

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=180,
    horizontal_flip=True,vertical_flip = True)

def model_builder(hp):
  model = Sequential()
  #model.add(Input(shape=(50,50,3)))
  for i in range(hp.Int('num_blocks', 1, 2)):
    hp_padding=hp.Choice('padding_'+ str(i), values=['valid', 'same'])
    hp_filters=hp.Choice('filters_'+ str(i), values=[32, 64])

    model.add(Conv2D(hp_filters, (3, 3), padding=hp_padding, activation='relu', kernel_initializer='he_uniform', input_shape=(50, 50, 3)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Dropout(hp.Choice('dropout_'+ str(i), values=[0.0, 0.1, 0.2])))
    model.add(Flatten())

    hp_units = hp.Int('units', min_value=25, max_value=150, step=25)
    model.add(Dense(hp_units, activation='relu', kernel_initializer='he_uniform'))
    model.add(Dense(10,activation="softmax"))
    hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])
    hp_optimizer=hp.Choice('Optimizer', values=['Adam', 'SGD'])

    if hp_optimizer == 'Adam':
      hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])

    elif hp_optimizer == 'SGD':
      hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])
      nesterov=True
      momentum=0.9

    model.compile(loss=keras.losses.binary_crossentropy, optimizer=tf.keras.optimizers.Adam(learning_rate=hp_learning_rate), metrics=['accuracy'])
    return model

perform the tuner search:

tuner_cnn = kt.tuners.BayesianOptimization(
    model_builder,
    objective='val_loss',
    max_trials=100,
    directory='.',
    project_name='tuning-cnn')

tuner_cnn.search(datagen.flow(X_trainRusReshaped,Y_trainRusHot), epochs=50, batch_size=256)

I also tried to do:

tuner_cnn.search(X_trainRusReshaped, Y_trainRusHot, epochs=80, validation_data=(X_testRusReshaped, Y_testRusHot), callbacks=[stop_early])

But it does not work neither. Any idea?

Answer

From the full error message I was able to narrow down where the issue is coming from. The issue is that your last Dense layer has 10 units, which means you expect 10 classes (you even chose the correct activation function given the number of units). However you have Binary CrossEntropy as loss.

So you either have 10 classes and use either categorical or sparse categorical CrossEntropy or you have 2 classes and so the loss is indeed Binary CrossEntropy.

Advertisement

Answer