I am doing a binary classification in Keras, using DenseNet.
Created weighted classes:
# Assign weights
weight_for_0 = num_normal/(num_normal + num_covid)
weight_for_1 = num_covid/(num_normal + num_covid)
class_weight = {0: weight_for_0, 1: weight_for_1}
# Print
print(f"Weight for class 0: {weight_for_0:.2f}")
print(f"Weight for class 1: {weight_for_1:.2f}")
As a result, I have
Weight for class 0: 0.74 Weight for class 1: 0.26
I fitted the model with class_weight
history_dense201_weighted = model_dense_201.fit_generator(train_generator, epochs = 20, validation_data = valid_generator, class_weight = class_weight, callbacks = [# mcp_save, early_stopping, tensorboard_callback])
But when I want to evaluate the model, I am not sure how to evaluate the weighted model, because the class_weight is a part of the history.
How to update this code, using instead of default model_dense_201 model a weighted model?
# Evaluation
evaluation = model_dense_201.evaluate(valid_generator)
print(f"Validation Accuracy: {evaluation[1] * 100:.2f}%")
evaluation = model_dense_201.evaluate(train_generator)
print(f"Train Accuracy: {evaluation[1] * 100:.2f}%")
Advertisement
Answer
Found this.
https://github.com/tensorflow/tensorflow/issues/35825
Quote from Francois (aka Chollet):
“The reason we don’t support class weights in evaluate is that the class_weight argument represents sample weights that are computed from the labels, but the labels should not be an input to the model during evaluation. During training this is fine, but during evaluation this represents a data leak from the labels to your metrics. If you used class weighting in evaluate, your score would not be reproducible on real test data (when you don’t have the labels).
So this is conceptually wrong.”