Incorporating Categorical Variables into Insurance Cost Models

Introduction and Lesson Overview

Welcome back to PredictHealth's Multi-Factor Cost Models course. In our previous lesson, you successfully built your first multiple regression model using numerical variables — age and BMI — to predict insurance costs. You learned how to prepare data, train models, and interpret coefficients in a business context. That model achieved solid performance, but we left some important information on the table. In real-world insurance pricing, some of the most significant cost drivers aren't numerical at all — they're categorical. Think about it: whether someone smokes or not can dramatically impact their health risks and insurance costs. Similarly, gender and geographic region can influence pricing due to different risk profiles and healthcare costs across areas. In this lesson, you will learn how to incorporate these categorical variables — smoking status, gender, and region — into your regression models. This means moving from a simple two-feature model to a comprehensive model that captures the full picture of what drives insurance costs. You'll discover how to prepare categorical data for machine learning, build sophisticated preprocessing pipelines, and interpret the results to make better business decisions. By the end of this lesson, you'll have a much more powerful and realistic insurance cost prediction model that PredictHealth could actually use in practice. Let's dive into the world of categorical variables and see how they transform our modeling capabilities.

Understanding Categorical Variables in Insurance Data

As a reminder, our insurance dataset contains several types of variables. In the previous lesson, we focused on the numerical ones: age and BMI . Now, let's examine the categorical features that we haven't used yet: sex, smoker, and region . These categorical variables represent different categories or groups rather than measurable quantities. The sex column contains values like "male" and "female," the smoker column has "yes" and "no," and the region column includes areas like "northeast," "southeast," "southwest," and "northwest." Why do these categories matter so much in insurance pricing? Each category represents a different risk profile. For example, statistical data show that smokers typically have higher healthcare costs due to smoking-related health issues. Geographic regions might have different healthcare costs, lifestyle factors, or even natural disaster risks that affect insurance claims. Gender can also correlate with different health patterns and life expectancy rates. The challenge is that machine learning algorithms, including linear regression, work with numbers, not text categories. When we fed age and BMI into our previous model, those were already numbers that the algorithm could use directly. But how do we handle "male" versus "female" or "yes" versus "no" for smoking status? This is where categorical data preprocessing becomes essential. Unlike numerical features, where the values have a natural mathematical relationship (age 30 is greater than age 25), categorical variables don't have this inherent ordering. We need a different approach to convert these categories into a format that our regression model can understand and use effectively.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Preparing Categorical Data with One-Hot Encoding

The solution to working with categorical data is called one-hot encoding. This technique converts each category into its own binary column, where 1 means the category applies and 0 means it doesn't.

Let's see how this works with a simple example. If we have a smoker column with values "yes" and "no," one-hot encoding creates two new columns: smoker_yes and smoker_no. For a person who smokes, smoker_yes would be 1 and smoker_no would be 0. For a non-smoker, it's the opposite.

However, there's an important detail here: we typically "drop the first" category to avoid redundancy. If we know that smoker_yes is 0, we automatically know that smoker_no must be 1. Keeping both columns would give our model redundant information and could cause mathematical problems. So we usually keep just smoker_yes — when it's 1, the person smokes; when it's 0, they don't.

Here's how we set up the preprocessing for our mixed data types using scikit-learn's ColumnTransformer:

Python

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

# Define which columns are numerical and which are categorical
numerical_features = ['age', 'bmi', 'children']
categorical_features = ['sex', 'smoker', 'region']

# Create preprocessing pipelines for both numerical and categorical data
preprocessor = ColumnTransformer(
    transformers=[
        ('num', 'passthrough', numerical_features),
        ('cat', OneHotEncoder(drop='first'), categorical_features)
    ])

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

# Define which columns are numerical and which are categorical
numerical_features = ['age', 'bmi', 'children']
categorical_features = ['sex', 'smoker', 'region']

# Create preprocessing pipelines for both numerical and categorical data
preprocessor = ColumnTransformer(
    transformers=[
        ('num', 'passthrough', numerical_features),
        ('cat', OneHotEncoder(drop='first'), categorical_features)
    ])

The ColumnTransformer allows us to apply different preprocessing steps to different types of columns. For numerical features, we use 'passthrough', which means they go through unchanged. For categorical features, we apply OneHotEncoder(drop='first') to convert them into the binary format our model needs.

Training and Evaluating the Enhanced Model

Let's train our comprehensive model and see how it performs compared to our previous numerical-only model:

Python

# Train the model
model_pipeline.fit(X_train, y_train)

# Make predictions
y_pred = model_pipeline.predict(X_test)

# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, y_pred)

print("Full Model with Categorical Variables:")
print(f"Mean Squared Error (MSE): {mse:.2f}")
print(f"Root Mean Squared Error (RMSE): {rmse:.2f}")
print(f"R-squared (R²): {r2:.2f}")

# Train the model
model_pipeline.fit(X_train, y_train)

# Make predictions
y_pred = model_pipeline.predict(X_test)

# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, y_pred)

print("Full Model with Categorical Variables:")
print(f"Mean Squared Error (MSE): {mse:.2f}")
print(f"Root Mean Squared Error (RMSE): {rmse:.2f}")
print(f"R-squared (R²): {r2:.2f}")

plaintext

Full Model with Categorical Variables:
Mean Squared Error (MSE): 33596915.85
Root Mean Squared Error (RMSE): 5796.28
R-squared (R²): 0.78

Full Model with Categorical Variables:
Mean Squared Error (MSE): 33596915.85
Root Mean Squared Error (RMSE): 5796.28
R-squared (R²): 0.78

Compare this to our previous model that used only age and BMI, which had an R² of about 0.15. The improvement to 0.75 means our model now explains 75% of the variation in insurance costs, compared to 15% before. This might seem like a small improvement, but in the insurance industry, even small improvements in prediction accuracy can translate to significant business value.

The lower RMSE also indicates that our predictions are more accurate on average. This enhanced accuracy comes from capturing the important categorical factors that influence insurance costs — factors that our numerical-only model completely missed.

Python

# Save the trained pipeline for future use
import joblib
joblib.dump(model_pipeline, 'insurance_cost_model.pkl')
print("Model saved successfully!")

# Save the trained pipeline for future use
import joblib
joblib.dump(model_pipeline, 'insurance_cost_model.pkl')
print("Model saved successfully!")

Once you've trained and validated your model, it's crucial to save it for future use. The joblib.dump() function saves the entire pipeline — including both the preprocessing steps and the trained regression model — into a single file. This means PredictHealth can load this saved model later to make predictions on new customers without having to retrain from scratch. To load the model later, simply use loaded_model = joblib.load('insurance_cost_model.pkl').

Interpreting Categorical Variable Coefficients

Understanding what our model learned requires us to examine the coefficients, but with categorical variables, this becomes more complex. After one-hot encoding, we have more features than we started with, and we need to extract their names properly.

Python

# Get the feature names after one-hot encoding
ohe = model_pipeline.named_steps['preprocessor'].transformers_[1][1]
cat_feature_names = ohe.get_feature_names_out(categorical_features)

# Get all feature names (numerical + categorical)
all_feature_names = numerical_features + list(cat_feature_names)

# Extract coefficients
coefficients = model_pipeline.named_steps['regressor'].coef_
intercept = model_pipeline.named_steps['regressor'].intercept_

# Print coefficients with feature names
print("\nModel Coefficients:")
for feature, coef in zip(all_feature_names, coefficients):
    print(f"{feature}: {coef:.2f}")
print(f"Intercept: {intercept:.2f}")

# Get the feature names after one-hot encoding
ohe = model_pipeline.named_steps['preprocessor'].transformers_[1][1]
cat_feature_names = ohe.get_feature_names_out(categorical_features)

# Get all feature names (numerical + categorical)
all_feature_names = numerical_features + list(cat_feature_names)

# Extract coefficients
coefficients = model_pipeline.named_steps['regressor'].coef_
intercept = model_pipeline.named_steps['regressor'].intercept_

# Print coefficients with feature names
print("\nModel Coefficients:")
for feature, coef in zip(all_feature_names, coefficients):
    print(f"{feature}: {coef:.2f}")
print(f"Intercept: {intercept:.2f}")

plaintext

Model Coefficients:
age: 256.98
bmi: 337.09
children: 425.28
sex_male: -18.59
smoker_yes: 23651.13
region_northwest: -370.68
region_southeast: -657.86
region_southwest: -809.80
Intercept: -11931.22

Model Coefficients:
age: 256.98
bmi: 337.09
children: 425.28
sex_male: -18.59
smoker_yes: 23651.13
region_northwest: -370.68
region_southeast: -657.86
region_southwest: -809.80
Intercept: -11931.22

The categorical coefficients tell us fascinating stories about insurance costs. The smoker_yes coefficient of 23,615.96 means that being a smoker increases insurance costs by approximately $23,616 compared to being a non-smoker, all else being equal. This massive impact demonstrates why smoking status is such a critical factor in insurance pricing.

The sex_male coefficient of 131.31 suggests that being male increases costs by about $131 compared to being female. The regional coefficients are all negative, which means that compared to the baseline region (northeast, which was dropped), all other regions have lower insurance costs.

Introduction and Lesson Overview

Understanding Categorical Variables in Insurance Data

Join the 1M+ learners on CodeSignal

Preparing Categorical Data with One-Hot Encoding

Building Comprehensive Modeling Pipelines

Training and Evaluating the Enhanced Model

Interpreting Categorical Variable Coefficients

Making Predictions with Customer Categories

Summary and Practice Preparation