Lesson Overview

In this final lesson on Model Evaluation Post-Optimization, we navigate through logistic regression, decision trees, and explore ensemble techniques to enhance accuracy. We tie it all together by comparing models post-optimization and making an informed selection to ensure reliable predictions.

Model Evaluation and Selection: A Practical Approach

Post-optimization model evaluation transcends mere accuracy comparisons. It involves analyzing performance metrics such as precision, recall, and f1-score to choose a model that truly understands the data, ensuring selection of a model that generalizes well on unseen data.

Logistic Regression Optimization

We begin by fine-tuning our Logistic Regression model:

from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression

# Hyperparameter tuning with GridSearchCV
lr_params = {'C': [0.001, 0.01, 0.1, 1, 10, 100], 'penalty': ['l2']}
lr = LogisticRegression(random_state=42, max_iter=1000)
clf_lr = GridSearchCV(lr, lr_params, cv=5)
clf_lr.fit(X_train, y_train)

# Evaluating optimized Logistic Regression
lr_pred = clf_lr.predict(X_test)
print("Logistic Regression Classification Report:\n", classification_report(y_test, lr_pred))

The best parameters found are {'C': 1, 'penalty': 'l2'}, significantly enhancing our model performance. Evaluating the optimized Logistic Regression yields:

Logistic Regression Classification Report:
               precision    recall  f1-score   support

           0       0.97      0.98      0.98        63
           1       0.99      0.98      0.99       108

    accuracy                           0.98       171
   macro avg       0.98      0.98      0.98       171
weighted avg       0.98      0.98      0.98       171
Random Forest & Gradient Boosting Performance

Next, we evaluate Random Forest and Gradient Boosting:

from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier

# Random Forest
rf = RandomForestClassifier(random_state=42).fit(X_train, y_train)
rf_pred = rf.predict(X_test)
print("Random Forest Accuracy:", accuracy_score(y_test, rf_pred))

# Gradient Boosting
gb = GradientBoostingClassifier(random_state=42).fit(X_train, y_train)
gb_pred = gb.predict(X_test)
print("Gradient Boosting Accuracy:", accuracy_score(y_test, gb_pred))
  • Random Forest Classifier Accuracy: 0.9707602339181286
  • Gradient Boosting Classifier Accuracy: 0.9590643274853801

These comparisons illuminate the balanced performance of Logistic Regression across all metrics, overtaking the ensemble techniques in our scenario.

Sign up
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal