Tuning L2 Regularization in Logistic Regression

In the previous lesson, you learned how to evaluate a classification model using the confusion matrix and classification report. These tools helped you identify patterns of underfitting or overfitting. Now it’s time to move from diagnosis to fixing. One of the most powerful tools for improving model performance is regularization — a technique that helps control model complexity and improve generalization. In this lesson, you’ll learn how to apply L2 regularization to logistic regression and use the C parameter to find the right balance between underfitting and overfitting.

In real-world applications like spam detection or medical diagnosis, tuning this parameter carefully can dramatically improve the model’s ability to generalize and avoid costly errors.

In scikit-learn’s LogisticRegression, regularization is enabled by default, and the strength of regularization is controlled by the C parameter. One common misconception is that a larger C means stronger regularization — but it’s actually the opposite.

• Smaller C = Stronger regularization → simpler model
• Larger C = Weaker regularization → more flexible model

L2 regularization works by adding a penalty to the loss function that discourages large coefficient values. The result is a model that favors simpler explanations and is less likely to overfit the training data.

Let’s train multiple logistic regression models using different values of C, and observe how regularization strength affects model performance.

from sklearn.linear_model import LogisticRegression
import numpy as np

# np.logspace(-4, 4, 10) generates 10 values spaced evenly on a log scale between 10^-4 and 10^4.
# This gives us a wide range of C values to test, from very strong to very weak regularization.
C_values = np.logspace(-4, 4, 10)
print(f"{'C':<12} {'Train Acc':<12} {'Test Acc':<12}")

for C in C_values:
    # solver='liblinear' is a good choice for small datasets and supports L2 regularization.
    model = LogisticRegression(C=C, penalty='l2', solver='liblinear')
    model.fit(X_train, y_train)
    train_acc = model.score(X_train, y_train)
    test_acc = model.score(X_test, y_test)
    print(f"{C:<12.4f} {train_acc:<12.4f} {test_acc:<12.4f}")

• np.logspace(-4, 4, 10) creates 10 values between 0.0001 (10^-4) and 10000 (10^4), spaced evenly on a logarithmic scale. This helps you explore a wide range of regularization strengths.
• solver='liblinear' specifies the algorithm used to fit the model. 'liblinear' is a good default for small datasets and supports L2 regularization.