How to hypertune input shape using keras tuner?

I am trying to hypertune the input shape of an LSTM model based on the different values of timesteps. However, I am facing an issue. While initializing the model, the default value of timesteps (which is 2) is chosen, and accordingly, the build_model.scaled_train is created of shape (4096, 2, 64). Thus the value of input_shape during initialization is (2, 64). When the training starts and the value of timesteps is arbitrarily chosen as 16, then build_model.scaled_train has shape (512, 16, 64). This means that input_shape now takes the value (16, 64). However, this is not reflected in the model. The InputLayer retains the shape (2, 64) it got during initialization. Hence, an error – Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 16, 64), found shape=(None, 2, 64).

def build_model(hp):
  timesteps = hp.Choice('timesteps', [2, 4, 8, 16], ordered = False)
  ....

  DFS, UFS = get_data_in_shape(DF, UF, timesteps)
  build_model.scaled_train, build_model.train_label = train_test_splitting(DFS, UFS)

  model = keras.Sequential()
  model.add(InputLayer(input_shape = (timesteps, nosamples))
  ...
  ...
  return model
  
class MyTuner(BayesianOptimization):
  def run_trial(self, trial, *args, **kwargs):
  kwargs['batch_size'] = trial.hyperparameters.Choice('batch_size', [32, 64, 128, 256])
  return super(MyTuner, self).run_trial(trial, *args, **kwargs)

tuner = MyTuner(
   build_model, 
   objective ='val_loss', 
   max_trials = 20,
   overwrite = True,
   directory = '/content/drive/MyDrive/Colab Notebooks', 
   project_name = 'bo4')

When I start hyperparameter tuning, this happens.

tuner.search(build_model.scaled_train, build_model.train_label, validation_split = 0.2, epochs = 100, callbacks = [early_stopping])

Error –

    Search: Running Trial #1

Value             |Best Value So Far |Hyperparameter
16                |?                 |timesteps
4                 |?                 |layers
1024              |?                 |unitsLSTM
0.15              |?                 |rate
64                |?                 |unitsANN
0.001             |?                 |learning_rate

Epoch 1/100
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-76-10e3851dd45f> in <module>()
----> 1 tuner.search(build_model.scaled_train, build_model.train_label, validation_split = 0.2, epochs = 100, callbacks = [early_stopping]) #,  model_checkpoint

6 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
   1145           except Exception as e:  # pylint:disable=broad-except
   1146             if hasattr(e, "ag_error_metadata"):
-> 1147               raise e.ag_error_metadata.to_exception(e)
   1148             else:
   1149               raise

ValueError: in user code:

    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 859, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility
        raise ValueError(f'Input {input_index} of layer "{layer_name}" is '

    ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 16, 64), found shape=(None, 2, 64)

I think I am making a logical mistake somewhere but cannot find it. Please help.

Answer

I made some changes which are written below and it worked fine. But I don’t know if it is the optimal solution.

def build_model(hp):
...
...
  scaled_train, train_label = train_test_splitting(DFS, UFS)
...
...
  return model, scaled_train, train_label

class MyTuner(BayesianOptimization):
  def run_trial(self, trial, *args, **kwargs):
    hp = trial.hyperparameters
    model, scaled_train, train_label = self.hypermodel.build(hp)
    kwargs['batch_size'] = trial.hyperparameters.Choice('batch_size', [32, 64, 128, 256])
    return self.hypermodel.fit(hp, model, scaled_traintrain, train_label, *args, **kwargs)

Advertisement

Answer