Making Predictions and Evaluating Performance

Welcome back! You've already taken a huge step by building and evaluating a logistic regression model in the previous lesson. Now, let's move forward and see how to make predictions using this model and evaluate its performance.

What You'll Learn

In this lesson, you will:

  1. Make predictions with your trained model using the test data.
  2. Evaluate the performance of the model using a confusion matrix.
  3. Understand what the evaluation results mean for your model.

By the end of this lesson, you will be able to:

  • Use the predict function in R to generate predictions from your logistic regression model.
  • Interpret a confusion matrix to understand the performance of your model.

You must be familiar with most of the code shown below from previous units. The prediction step, added here, will be our focus in this lesson:

# Load the mtcars dataset
data(mtcars)

# Set seed for reproducibility
set.seed(123)

# Convert categorical columns to factors
mtcars$am <- as.factor(mtcars$am)
mtcars$cyl <- as.factor(mtcars$cyl)
mtcars$vs <- as.factor(mtcars$vs)
mtcars$gear <- as.factor(mtcars$gear)
mtcars$carb <- as.factor(mtcars$carb)

# Splitting data into training and testing sets
trainIndex <- createDataPartition(mtcars$am, p = 0.7, list = FALSE, times = 1)
trainData <- mtcars[trainIndex,]
testData <- mtcars[-trainIndex,]

# Feature scaling (excluding factor columns)
numericColumns <- sapply(trainData, is.numeric)
preProcValues <- preProcess(trainData[, numericColumns], method = c("center", "scale"))
trainData[, numericColumns] <- predict(preProcValues, trainData[, numericColumns])
testData[, numericColumns] <- predict(preProcValues, testData[, numericColumns])

# Train a logistic regression model
model <- train(am ~ mpg, data = trainData, method = "glm", family = "binomial")

# Making predictions
predictions <- predict(model, testData)

# Evaluating the model
confusion <- confusionMatrix(predictions, testData$am)
print(confusion)

Output:

Confusion Matrix and Statistics

          Reference
Prediction 0 1
         0 4 0
         1 1 3
                                          
               Accuracy : 0.875           
                 95% CI : (0.4735, 0.9968)
    No Information Rate : 0.625           
    P-Value [Acc > NIR] : 0.135           
                                          
                  Kappa : 0.75            
                                          
 Mcnemar's Test P-Value : 1.000           
                                          
            Sensitivity : 0.800           
            Specificity : 1.000           
         Pos Pred Value : 1.000           
         Neg Pred Value : 0.750           
             Prevalence : 0.625           
         Detection Rate : 0.500           
   Detection Prevalence : 0.500           
      Balanced Accuracy : 0.900           
                                          
       'Positive' Class : 0
Sign up
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal