Understanding the Confusion Matrix, Precision, and Recall in Classification Metrics

Welcome! Today, we are peeling back the layers of classification metrics, notably the confusion matrix, precision, and recall. This lesson delves into their theory and provides a practical illustration in Python.

The performance of binary classifiers is evaluated by comparing predicted and actual values; this structure is encoded as a confusion matrix. A confusion matrix produces four outcomes:

1. True Positive (TP): Correct positive prediction.
2. True Negative (TN): Correct negative prediction.
3. False Positive (FP): Incorrect positive prediction.
4. False Negative (FN): Incorrect negative prediction.

Consider an email spam filter, classifying Spam (positive) and Not Spam (negative) as follows:

We'll assemble a confusion matrix using a binary classification:

import numpy as np

true_labels = np.array([0, 0, 1, 1, 0, 1, 0, 1, 1, 1])
predicted_labels = np.array([0, 1, 0, 1, 0, 1, 1, 1, 1, 0])

TP = np.sum((predicted_labels == 1) & (true_labels == 1))
TN = np.sum((predicted_labels == 0) & (true_labels == 0))
FP = np.sum((predicted_labels == 1) & (true_labels == 0))
FN = np.sum((predicted_labels == 0) & (true_labels == 1))

print("Confusion Matrix:\n TP: ", TP, "\tFP: ", FP, "\n FN: ", FN, "\tTN: ", TN)

'''Output:
Confusion Matrix:
 TP:  4 	FP:  2 
 FN:  2 	TN:  2
'''

The code uses numpy's bitwise "&" operator to perform element-wise comparison between the predicted_labels and true_labels arrays. It then uses numpy's sum function to count the number of True values in the resulting comparison, and assigns these counts to the TP, TN, FP, FN variables.