trying to callibrate keras model

I’m trying to calibrate my CNN model by Sklearn implementation CalibratedClassifierCV, tried to wrap it as KerasClassifier and to override the predict function but without success. someone could say me what I did wrong? this is the model code:

def create_model():
    model = Sequential()
    model.add(Conv2D(64, kernel_size=(3,3), activation = 'relu', input_shape=(28, 28 ,1) ))
    model.add(MaxPooling2D(pool_size = (2, 2)))

    model.add(Conv2D(64, kernel_size = (3, 3), activation = 'relu'))
    model.add(MaxPooling2D(pool_size = (2, 2)))

    model.add(Conv2D(64, kernel_size = (3, 3), activation = 'relu'))
    model.add(MaxPooling2D(pool_size = (2, 2)))

    model.add(Flatten())
    model.add(Dense(128, activation = 'relu'))
    model.add(Dropout(0.20))

    model.add(Dense(24, activation = 'softmax'))

    model.compile(loss = keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adam(), metrics=['accuracy'])
    
    return model

this is me trying to calibrate it :

model = KerasClassifier(build_fn=create_model,epochs=5, batch_size=128,validation_data=(evalX_cnn, eval_y_cnn))
model.fit(trainX_cnn, train_y_cnn)
model_c = CalibratedClassifierCV(base_estimator=model, cv='prefit')
model_c.fit(valX_cnn, val_y_cnn)

the output :

-------------------------------------------------------
TypeError             Traceback (most recent call last)
<ipython-input-19-3d3ce9ce4fca> in <module>
----> 1 model_c.fit(np.array(valX_cnn), np.array(val_y_cnn))

~anaconda3libsite-packagessklearncalibration.py in fit(self, X, y, sample_weight)
    286             pred_method, method_name = _get_prediction_method(base_estimator)
    287             n_classes = len(self.classes_)
--> 288             predictions = _compute_predictions(pred_method, method_name, X, n_classes)
    289 
    290             calibrated_classifier = _fit_calibrator(

~anaconda3libsite-packagessklearncalibration.py in _compute_predictions(pred_method, method_name, X, n_classes)
    575         (X.shape[0], 1).
    576     """
--> 577     predictions = pred_method(X=X)
    578 
    579     if method_name == "decision_function":

TypeError: predict_proba() missing 1 required positional argument: 'x'

valX_cnn and val_y_cnn are of type np.array.

tried even to override the method:

keras.models.Model.predict_proba = keras.models.Model.predict

Answer

The problem is because predict_proba from KerasClassifier requires x as input while predict_proba method from sklearn accepts X as input argument (note the difference: X is not x).

You can simply overdrive the problem wrapping KerasClassifier into a new class to correct the predict_proba method.

samples,classes = 100,3

X = np.random.uniform(0,1, (samples,28,28,1))
Y = tf.keras.utils.to_categorical(np.random.randint(0,classes, (samples)))

class MyKerasClassifier(KerasClassifier):
  
  def predict_proba(self, X):
      return self.model.predict(X)

model = MyKerasClassifier(build_fn=create_model, epochs=3, batch_size=128)
model.fit(X, Y)

model_c = CalibratedClassifierCV(base_estimator=model, cv='prefit')
model_c.fit(X, Y)

Advertisement

Answer