Fine Tuning Prediction Models

Welcome back to the PredictHealth's Advanced Pricing System course. In the previous lessons, you learned how to model non-linear cost patterns using polynomial regression and how to capture feature interactions to improve the accuracy of your health insurance pricing models. These skills are essential for building models that reflect the real-world complexity of insurance data. In this lesson, we will take your modeling skills to the next level by focusing on fine-tuning prediction models.

The main goal of this lesson is to help you understand how to optimize your models for better performance and interpretability. You will learn how to establish a baseline using linear regression, then use regularization techniques like Ridge and Lasso regression to prevent overfitting and select important features. You will also see how to use tools like GridSearchCV to find the best model parameters, compare model performances, and visualize the results. By the end of this lesson, you will be able to fine-tune regression models for health insurance pricing and understand the trade-offs involved in model selection.

Before diving into model tuning, it's crucial to understand the bias-variance tradeoff. This fundamental concept explains why regularization is necessary and helps you choose the right model complexity.

Bias refers to how far off your model's predictions are from the true values on average. Simple models like linear regression tend to have high bias because they may miss important patterns in the data. Variance refers to how much your model's predictions would change if you trained it on different datasets. Complex models tend to have high variance because they're sensitive to small changes in the training data.

The goal is to find the sweet spot where both bias and variance are reasonably low. Regularization techniques like Ridge and Lasso help achieve this balance by controlling model complexity:

• Ridge regression reduces variance by shrinking coefficients but maintains all features
• Lasso regression reduces both variance and bias by eliminating irrelevant features entirely

To begin fine-tuning, establish a baseline model using linear regression. After preparing your data (splitting, standardizing, and encoding categorical variables), fit the model and evaluate its performance:

# Prepare the data 
insurance_data = pd.read_csv('datasets/insurance.csv')

X = pd.get_dummies(insurance_data.drop('charges', axis=1), drop_first=True)
y = insurance_data['charges']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Fit linear regression
lr = LinearRegression()
lr.fit(X_train_scaled, y_train)
lr_pred = lr.predict(X_test_scaled)
lr_rmse = np.sqrt(mean_squared_error(y_test, lr_pred))
lr_r2 = r2_score(y_test, lr_pred)

print(f"Linear Regression - RMSE: {lr_rmse:.2f}, R²: {lr_r2:.4f}")

Output:

Linear Regression - RMSE: 5796.28, R²: 0.7836

This baseline helps you measure improvements from regularization techniques.

Ridge regression prevents overfitting by adding a penalty proportional to the sum of squared coefficients. The alpha parameter controls this penalty strength:

• Low alpha (0.01-0.1): Light regularization, similar to linear regression
• Medium alpha (1.0-10.0): Moderate regularization, good balance for most cases
• High alpha (100+): Strong regularization, heavily shrinks coefficients

Note: GridSearchCV is designed to maximize the provided scoring metric. Since we want to minimize the mean squared error (MSE), we use 'neg_mean_squared_error' as the scoring parameter. This way, GridSearchCV selects the model with the lowest MSE by maximizing its negative value. After cross-validation, best_estimator_ is automatically retrained on the full training dataset using the best parameters found during the search.

ridge_params = {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0]}
ridge_cv = GridSearchCV(Ridge(), ridge_params, scoring='neg_mean_squared_error', cv=5)
ridge_cv.fit(X_train_scaled, y_train)
best_ridge = ridge_cv.best_estimator_
ridge_pred = best_ridge.predict(X_test_scaled)
ridge_rmse = np.sqrt(mean_squared_error(y_test, ridge_pred))
ridge_r2 = r2_score(y_test, ridge_pred)

print(f"Ridge Regression (alpha={best_ridge.alpha}) - RMSE: {ridge_rmse:.2f}, R²: {ridge_r2:.4f}")

Output:

Ridge Regression (alpha=10.0) - RMSE: 5803.95, R²: 0.7830

Lasso regression performs both regularization and automatic feature selection by setting some coefficients exactly to zero. This creates simpler, more interpretable models. Like Ridge, GridSearchCV retrains the best_estimator_ on the full training data after finding optimal parameters:

lasso_params = {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0]}
lasso_cv = GridSearchCV(Lasso(), lasso_params, scoring='neg_mean_squared_error', cv=5)
lasso_cv.fit(X_train_scaled, y_train)
best_lasso = lasso_cv.best_estimator_
lasso_pred = best_lasso.predict(X_test_scaled)
lasso_rmse = np.sqrt(mean_squared_error(y_test, lasso_pred))
lasso_r2 = r2_score(y_test, lasso_pred)

print(f"Lasso Regression (alpha={best_lasso.alpha}) - RMSE: {lasso_rmse:.2f}, R²: {lasso_r2:.4f}")

Output:

Lasso Regression (alpha=100.0) - RMSE: 5835.80, R²: 0.7806

One of Lasso's key advantages is feature selection. You can examine which features the model considers most important:

# Get feature names
feature_names = X.columns

# Create feature importance DataFrame
feature_importance = pd.DataFrame({
    'feature': feature_names,
    'coefficient': best_lasso.coef_
})

# Show selected features (non-zero coefficients)
selected_features = feature_importance[feature_importance['coefficient'] != 0]
selected_features = selected_features.sort_values('coefficient', key=abs, ascending=False)
print("Selected Features by Lasso:")
print(selected_features)

Output:

Selected Features by Lasso:
            feature  coefficient
4        smoker_yes  9453.068030
0               age  3528.802468
1               bmi  1892.789994
2          children   424.978753
7  region_southwest  -104.379764
6  region_southeast   -15.589796

This helps you understand which factors most influence insurance charges and can guide business decisions.

Choose your model based on your specific goals:

Use Linear Regression when:

• You have few features and want maximum interpretability
• You need to understand the effect of each feature clearly
• Overfitting is not a concern

Use Ridge Regression when:

• You have many correlated features
• You want to keep all features but reduce overfitting
• You need stable coefficient estimates

Use Lasso Regression when:

• You have many features and suspect some are irrelevant
• You want automatic feature selection
• You need a simpler, more interpretable model
• You're dealing with high-dimensional data

In this lesson, you learned how to fine-tune prediction models for health insurance pricing using the bias-variance tradeoff as your guiding principle. You established baselines with linear regression, then used Ridge and Lasso regression to improve model performance and interpretability. You also learned how to examine feature importance, choose the right model for your needs, and understand the practical implications of different regularization strengths.

These skills are essential for building robust and interpretable pricing models in the insurance industry. In the next section, you will practice these techniques yourself, building confidence in your ability to fine-tune and select the best models for real-world data.