Classifying Text with Logistic Regression in Python

Classifying Text with Logistic Regression in Python | CodeSignal Learn

Lesson Overview and Goal

Hello and welcome back! Today, we are diving into the world of text classification using Logistic Regression . In some of our previous lessons, we discussed various aspects of natural language processing, such as vectorization of text data. Today, we are going to build upon those lessons and use Logistic Regression to classify messages. By the end of this lesson you will have a firm grasp on logistic regression, how to use it for text classification and evaluate the prediction accuracy.

Understanding Logistic Regression

y = \frac{1}{1 + e^{-x}}

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Before we start coding, let's delve a bit deeper into what Logistic Regression is, focusing specifically on its role in classifying text messages, like identifying spam. Logistic Regression is a go-to Machine Learning algorithm for binary classification tasks—it's how we decide if a message is spam (1) or not spam (0), based on the content of the message.

While Logistic Regression might sound complex, especially when we introduce formulas, it's grounded in a simple yet powerful idea. It uses what's called the logistic (or sigmoid) function, which helps predict probabilities that are specifically between 0 and 1, perfectly matching our need to categorize messages into two distinct groups. The heart of this function is a formula:

$y = \frac{1}{1 + e^{-x}}$

Think of $x$ as the essence of a text message turned into numbers using vectorization, and $e$ is a constant used in math for calculating rates of change. The formula outputs $y$ , which we can interpret as the chance of a message being spam.

Now, let's simplify how this works: Imagine our Logistic Regression model as a very keen observer. Each time it looks at a message, it uses this formula to calculate a score ( $y$ )—think of it as the "spamminess" level of the message. If the score is above a certain threshold, usually 0.5, our keen observer labels the message as spam; if it's below, the message is considered not spam.

The magic of Logistic Regression is that over time, with exposure to more messages and information on whether those messages were actually spam, it adjusts its understanding to predict more accurately. So, each message helps it learn and get better at spotting spam. It's like teaching it to distinguish between different types of messages based on their "spamminess" level, gradually making it an expert spam detector with just a simple mathematical formula.

Building on what you've learned about loading datasets, message vectorization, and splitting data into training and testing sets, we can proceed directly with the logistic regression implementation. To create and train our logistic regression model, we use the LogisticRegression class from sklearn.linear_model . Python from sklearn.linear_model import LogisticRegression log_reg = LogisticRegression() log_reg.fit(X_train, y_train) from sklearn.linear_model import LogisticRegression log_reg = LogisticRegression() log_reg.fit(X_train, y_train) After training, the model is ready to make predictions on the test data. Python y_pred = log_reg.predict(X_test) y_pred = log_reg.predict(X_test) With these steps, you've successfully implemented logistic regression for classifying your text data.

Evaluating Logistic Regression Model

So, we have our model and it's making predictions, but how do we know if it's any good? To measure the classification performance of our model, we use metrics. There are numerous metrics available, such as accuracy, precision, recall, F1-score, but for this lesson, we will use accuracy score. Accuracy score is a commonly used metric that gives us the proportion of correct classifications our model made. For example, if our model correctly classified 84 out of 100 messages, the accuracy score is 0.84. The accuracy score can be calculated using accuracy_score from sklearn.metrics . Python from sklearn.metrics import accuracy_score print("Accuracy:", accuracy_score(y_test, y_pred)) from sklearn.metrics import accuracy_score print("Accuracy:", accuracy_score(y_test, y_pred)) The output will be: text Accuracy: 0.964824120603015 Accuracy: 0.964824120603015 This indicates that our logistic regression model was able to correctly classify approximately 96.48% of the messages in the test set, which is a strong result, demonstrating the effectiveness of logistic regression for text classification tasks. However, keep in mind that "accuracy" might not always be the best metric to evaluate performance, especially for imbalanced datasets where one class far outnumber the others. In such case, evaluation metrics like precision, recall, or F1-score can be more informative.

Lesson Overview and Goal

Understanding Logistic Regression

Join the 1M+ learners on CodeSignal

Implementation of Logistic Regression for Text Classification

Evaluating Logistic Regression Model

Takeaways and Real-world Applications

Lesson Summary and Practice