I am using an U-Net for segmenting my data of interest. The masks are grayscale and of size (256,256,1). There are 80 images in the test set. The test images (X_ts) and their respective ground-truth masks (Y_ts) are constructed, saved, and loaded like this:
from tqdm import tqdm im_width = 256 im_height = 256 ids_test = next(os.walk("data/test/image"))[2] print("No. of images = ", len(ids_test)) X_ts = np.zeros((len(ids_test), im_height, im_width, 3), dtype=np.float32) Y_ts = np.zeros((len(ids_test), im_height, im_width, 1), dtype=np.float32) print(X_ts.shape) print(Y_ts.shape) for n, id_ in tqdm(enumerate(ids_test), total=len(ids_test)): # Load images img = load_img("data/test/image/"+id_, color_mode = "rgb") x_img = img_to_array(img) x_img = resize(x_img, (256,256,3), mode = 'constant', preserve_range = True) # Load masks mask = img_to_array(load_img("data/test/label/"+id_, color_mode = "grayscale")) mask = resize(mask, (256,256,1), mode = 'constant', preserve_range = True) # Save images X_ts[n] = x_img/255.0 Y_ts[n] = mask/255.0 np.save('./data/X_ts.npy',X_ts) np.save('./data/Y_ts.npy',Y_ts) #load the data X_ts = np.load('./data/X_ts.npy') Y_ts = np.load('./data/Y_ts.npy')
The shape of Y_ts (ground truth) is therefore (80,256,256,1) and these are of type “Array of type float32”. I predicted the masks using the trained model like this:
Y_ts_pred = model.predict(X_ts,batch_size=1) print(Y_ts_pred.shape) threshold=0.5 Y_ts_pred[Y_ts_pred<threshold]=0 Y_ts_pred[Y_ts_pred>=threshold]=1
The shape of Y_ts_pred is (80,256,256,1) and these are of type “Array of type float32”. Now, I compute the precision recall curves using the Y_ts (ground truth masks) and Y_ts_pred (predicted masks) like this:
from sklearn.metrics import precision_recall_curve import numpy as np import matplotlib.pyplot as plt precision, recall, thresholds = precision_recall_curve(Y_ts, Y_ts_pred) precision = np.fliplr([precision])[0] #to avoid getting negative AUC) recall = np.fliplr([recall])[0] #to avoid getting negative AUC) AUC_prec_rec = np.trapz(precision,recall) print "nArea under Precision-Recall curve: " +str(AUC_prec_rec) prec_rec_curve = plt.figure() plt.plot(recall,precision,'-',label='Area Under the Curve (AUC = %0.4f)' % AUC_prec_rec) plt.title('Precision - Recall curve') plt.xlabel("Recall") plt.ylabel("Precision") plt.legend(loc="lower right") plt.savefig("Precision_recall.png")
I get the following error:
File "/tmp/ipykernel_2425796/385729104.py", line 1, in <module> precision, recall, thresholds = precision_recall_curve(Y_ts, Y_ts_pred) File "/home/anaconda3/envs/tf262/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 859, in precision_recall_curve y_true, probas_pred, pos_label=pos_label, sample_weight=sample_weight File "/home/anaconda3/envs/tf262/lib/python3.7/site-packages/sklearn/metrics/_ranking.py", line 731, in _binary_clf_curve raise ValueError("{0} format is not supported".format(y_type)) ValueError: unknown format is not supported
Advertisement
Answer
I have to convert my ground truth and predicted masks into one-dimensional vectors and binarize them using a threshold value like below. Then, the above code will work.
Y_ts_pred = Y_ts_pred.reshape(Y_ts_pred.shape[0]*Y_ts_pred.shape[1]*Y_ts_pred.shape[2], 1) print(Y_ts_pred.shape) Y_ts = Y_ts.reshape(Y_ts.shape[0]*Y_ts.shape[1]*Y_ts.shape[2], 1) print(Y_ts.shape) Y_ts_pred = np.where(Y_ts_pred>0.5, 1, 0) Y_ts = np.where(Y_ts>0.5, 1, 0)