I am using precision_score in sklearn to evaluate the result of the outlier detection algorithm. I trained with one class only and predict on unseen data. So the label for the one class is just 0 all the way.
I have found the following:
There are two columns, truth and predicted. (I used the label encoder to beautify the number, in Local Outlier Factor, it output 1 for inlier and -1 for the outlier, I use label encoder to encode them into 0s and 1s, same for the truth)
However, the algorithm returns that my accuracy is 1, but precision is 0. It can be clearly seen that the predicted match with the truth completely. I would expect to get scores of 1s for both parameters. It comes with the below warning:
What should I do or any links I should be reading to mitigate this issue.
Advertisement
Answer
The documentation explains that with only two classes, it treats it as a binary problem. Precision is about true positives (guessing 1 when the answer is 1). You don’t have any—just true negatives (guessing 0 when the answer is 0).
If you’re really unhappy with that outcome, you can use the zero_division argument:
precision_score(truth, predicted, zero_division=1)
That way, you’ll get the 1 you want.