Logistic Regression with the Iris Dataset | CodeSignal Learn

Logistic Regression with the Iris Dataset | CodeSignal Learn

Lesson Overview

Welcome back! In this session, we're going to delve into the realm of Logistic Regression. Our primary materials for this exercise are Python, the Scikit-learn library, and the renowned Iris dataset. The end goal is the mastery of the theory and practical implementation of logistic regression for classification tasks and a solid comprehension of its utility and applications.

Understanding Logistic Regression

S(x) = \frac{1}{1 + e^{-x}}

Data Preprocessing

Before jumping into the modeling process, we need to ensure our dataset is structured and clean. We'll be using the Iris dataset, composed of measurements for numerous flowers from each of three species of Iris: Iris setosa, Iris virginica, and Iris versicolor. Let's load and inspect it: Python # Importing the iris dataset from sklearn from sklearn.datasets import load_iris # Loading the dataset iris = load_iris() # Printing a description print(iris.DESCR) The output of the code will provide a full description of the Iris dataset, including its features, number of instances, and summary statistics. Once loaded and understood, we need to partition the data into a training set, which the model will learn from, and a test set, used to evaluate the model's accuracy: Python from sklearn.model_selection import train_test_split # Splitting the data into a training set and a test set X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Implementing Logistic Regression

With data in place, it's time to initialize a logistic regression model. Scikit-learn simplifies this process, providing robust pseudo-English ways to implement most machine learning models, including logistic regression: from sklearn.linear_model import LogisticRegression # Instantiating a Logistic Regression model lr_model = LogisticRegression(solver='lbfgs', max_iter=200) # Fitting the model with our training data lr_model.fit(X_train, y_train) This process creates a logistic regression model and trains it with our features (X_train) and our target variable (y_train). Let's break down the two parameters we specified: solver='lbfgs' : This tells the model which optimization algorithm to use for finding the best parameters. lbfgs (Limited-memory Broyden-Fletcher-Goldfarb-Shanno) is the default solver in modern scikit-learn and works well for most problems, including multiclass classification like our Iris dataset with three species. max_iter=200 : This sets the maximum number of iterations the solver takes to converge on a solution. Increasing this from the default of 100 helps avoid convergence warnings, especially with more complex datasets. Other compatible solvers you may encounter include newton-cg, newton-cholesky, sag, and saga . Each has its own strengths depending on the dataset size and structure.

Interpreting Model Outcomes

Once fitted, we can use the model to predict the outcome for our test data: # Generating predictions predictions = lr_model.predict(X_test) The predict() function returns the predicted classes according to the fitted model. By comparing the predicted output to the actual values, we can understand how well the model performed. # Comparing predictions with actual values print("Predictions: ", predictions) print("Actual: ", y_test) output: Predictions: [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0] Actual: [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0] The output of the code consists of arrays of predicted and actual target values for comparison.

Evaluating the Logistic Regression Model

After training the Logistic Regression model, it's important to evaluate its performance to understand how well it's likely to perform on unseen data. One straightforward method for this is using the score method provided by Scikit-learn. This method returns the mean accuracy on the given test data and labels - in simpler terms, it tells you the proportion of correctly predicted instances. Here's how you can use it: # Evaluating the model on the training set train_accuracy = lr_model.score(X_train, y_train) print("Training Accuracy:", train_accuracy) # Evaluating the model on the test set test_accuracy = lr_model.score(X_test, y_test) print("Test Accuracy:", test_accuracy) output: Training Accuracy: 0.9666666666666667 Test Accuracy: 1.0 Interpreting the Results: Training Accuracy: This is the accuracy of the model on the data it was trained on. A high training accuracy typically indicates that the model has learned the training data well. Test Accuracy: This is the accuracy of the model on the test data, which it has not seen during training. This is a more robust indicator of how well the model will perform on new, unseen data. It's crucial to compare these two metrics. If your training accuracy is much higher than your test accuracy, it might be a sign that the model is overfitting to the training data. Conversely, if both accuracies are low, the model might be under-fitting.

Multiclass Classification in Logistic Regression

You might notice that the Iris dataset has three classes (setosa, versicolor, and virginica), while logistic regression was originally designed for binary (two-class) classification. So how does it handle three classes? In modern scikit-learn, when you use a solver like lbfgs, the model automatically uses multinomial logistic regression for problems with more than two classes. This approach models all three classes simultaneously, estimating the probability that a given flower belongs to each species and picking the most likely one. There is also an alternative approach called One-vs-Rest (OvR), which trains a separate binary classifier for each class (e.g., setosa vs. not-setosa, versicolor vs. not-versicolor, etc.) and combines the results. If you specifically want to use the OvR strategy, you can do so explicitly with scikit-learn's OneVsRestClassifier wrapper: from sklearn.multiclass import OneVsRestClassifier from sklearn.linear_model import LogisticRegression ovr_model = OneVsRestClassifier(LogisticRegression(solver='liblinear', max_iter=200)) ovr_model.fit(X_train, y_train) print("OvR Test Accuracy:", ovr_model.score(X_test, y_test)) For most use cases, the default multinomial approach with lbfgs works well and is the recommended starting point.

Python

from sklearn.linear_model import LogisticRegression

# Instantiating a Logistic Regression model
lr_model = LogisticRegression(solver='lbfgs', max_iter=200)

# Fitting the model with our training data
lr_model.fit(X_train, y_train)

Python

# Evaluating the model on the training set
train_accuracy = lr_model.score(X_train, y_train)
print("Training Accuracy:", train_accuracy)

# Evaluating the model on the test set
test_accuracy = lr_model.score(X_test, y_test)
print("Test Accuracy:", test_accuracy)

Python

from sklearn.multiclass import OneVsRestClassifier
from sklearn.linear_model import LogisticRegression

ovr_model = OneVsRestClassifier(LogisticRegression(solver='liblinear', max_iter=200))
ovr_model.fit(X_train, y_train)
print("OvR Test Accuracy:", ovr_model.score(X_test, y_test))