Understanding the Confusion Matrix, Precision, and Recall in Classification Metrics

Welcome! Today, we are peeling back the layers of classification metrics, notably the confusion matrix, precision, and recall. This lesson delves into their theory and provides a practical illustration in C++.

The performance of binary classifiers is evaluated by comparing predicted and actual values; this structure is encoded as a confusion matrix. A confusion matrix produces four outcomes:

1. True Positive (TP): Correct positive prediction.
2. True Negative (TN): Correct negative prediction.
3. False Positive (FP): Incorrect positive prediction.
4. False Negative (FN): Incorrect negative prediction.

Consider an email spam filter, classifying Spam (positive) and Not Spam (negative) as follows:

We'll assemble a confusion matrix using a binary classification:

#include <iostream>
#include <vector>
#include <algorithm>
using namespace std;

int main() {
    // Step 1: Get the data set
    vector<int> true_labels = {0, 0, 1, 1, 0, 1, 0, 1, 1, 1};
    vector<int> predicted_labels = {0, 1, 0, 1, 0, 1, 1, 1, 1, 0};

    // Step 2: Calculate confusion matrix components
    int TP = 0, TN = 0, FP = 0, FN = 0;

    for (size_t i = 0; i < true_labels.size(); ++i) {
        if (predicted_labels[i] == 1 && true_labels[i] == 1) {
            TP++;
        } else if (predicted_labels[i] == 0 && true_labels[i] == 0) {
            TN++;
        } else if (predicted_labels[i] == 1 && true_labels[i] == 0) {
            FP++;
        } else if (predicted_labels[i] == 0 && true_labels[i] == 1) {
            FN++;
        }
    }

    cout << "Confusion Matrix:\n TP: " << TP << "\tFP: " << FP
         << "\n FN: " << FN << "\tTN: " << TN << endl;

    return 0;
}

/*Output:
Confusion Matrix:
 TP:  4 	FP:  2
 FN:  2 	TN:  2
*/

The code uses a simple loop to perform element-wise comparison between the predicted_labels and true_labels vectors. It then counts the number of matches for each category and assigns these counts to the TP, TN, FP, FN variables.

We use the confusion matrix variables to calculate precision and recall:

#include <iostream>
#include <vector>
#include <cmath>
using namespace std;

double calculate_precision(int TP, int FP) {
    if (TP + FP == 0) return 0.0;
    return static_cast<double>(TP) / (TP + FP);
}

double calculate_recall(int TP, int FN) {
    if (TP + FN == 0) return 0.0;
    return static_cast<double>(TP) / (TP + FN);
}

int main() {
    int TP = 4, FP = 2, FN = 2;

    double precision = calculate_precision(TP, FP);
    double recall = calculate_recall(TP, FN);

    cout << "Precision: " << round(precision * 100) / 100.0 << endl;  // 0.67
    cout << "Recall: " << round(recall * 100) / 100.0 << endl;  // 0.67

    return 0;
}

Our C++ script defines two functions: calculate_precision and calculate_recall. These return precision and recall, respectively. Finally, we print the values of precision and recall.

The confusion matrix, precision, and recall form the foundation for performance measurement in classification tasks. They help us understand our model's functionality, which is becoming vital in real-world applications. For instance, in medical or spam classification scenarios, emphasis may shift between precision and recall depending on the specific evaluation aspect.

Congratulations! You've untangled the mysteries of the Confusion Matrix, Precision, and Recall metrics and their implementation in C++. Let's get to practice!