Hello! Today, we are diving into the world of data scaling techniques. Imagine you are playing a game where you need to fit different shapes into matching holes. If your shapes vary greatly in size, it can be challenging. Similarly, in data analysis and machine learning, features (or columns) in your dataset may have vastly different scales. This can affect the performance of your analysis or model.

Our goal for this lesson is to understand two key data scaling techniques: Standard Scaling and Min-Max Scaling. By the end of this lesson, you'll be able to apply these techniques to scale features in a dataset, making them easier to work with.

Let's use the Titanic dataset to perform Standard Scaling on the age and fare columns.

import pandas as pd
import seaborn as sns

# Load the Titanic dataset
titanic = sns.load_dataset('titanic')

# Calculate mean and standard deviation for 'age' and 'fare'
age_mean = titanic['age'].mean()
age_std = titanic['age'].std()
fare_mean = titanic['fare'].mean()
fare_std = titanic['fare'].std()

# Standard Scaling
titanic['age_standard'] = (titanic['age'] - age_mean) / age_std
titanic['fare_standard'] = (titanic['fare'] - fare_mean) / fare_std

print(titanic[['age', 'age_standard', 'fare', 'fare_standard']].head())

Output:

    age  age_standard     fare  fare_standard
0  22.0     -0.530005   7.2500      -0.502163
1  38.0      0.571430  71.2833       0.786404
2  26.0     -0.254646   7.9250      -0.488580
3  35.0      0.364911  53.1000       0.420494
4  35.0      0.364911   8.0500      -0.486064

Note: The Titanic age column contains missing values. Pandas’ mean() and std() skip missing values by default, so the scaling parameters are calculated only from non-missing ages. Missing ages will remain NaN after scaling. Also, Series.std() uses the sample standard deviation by default, equivalent to ddof=1, which can differ slightly from tools like scikit-learn’s StandardScaler, which uses population standard deviation.