Data Scaling Techniques

Hello! Today, we are diving into the world of data scaling techniques. Imagine you are playing a game where you need to fit different shapes into matching holes. If your shapes vary greatly in size, it can be challenging. Similarly, in data analysis and machine learning, features (or columns) in your dataset may have vastly different scales. This can affect the performance of your analysis or model.

Our goal for this lesson is to understand two key data scaling techniques: Standard Scaling and Min-Max Scaling. By the end of this lesson, you'll be able to apply these techniques to scale features in a dataset, making them easier to work with.

Standard Scaling is like leveling the playing field for your data. It transforms your data so it has a mean (average) of 0 and a standard deviation (how spread out the numbers are) of 1. This is especially useful when you want your data to follow a standard normal distribution.

The formula for standard scaling is:

$z = \frac{(X - \mu)}{\sigma}$

Where:

• $X$ is the original value.
• $\mu$ is the mean of the values.
• $\sigma$ is the standard deviation of the values.

In simpler terms, you subtract the average value from each data point and then divide by how much your data varies from the average.

Let's use the Titanic dataset to perform Standard Scaling on the age and fare columns.

import pandas as pd
import seaborn as sns

# Load the Titanic dataset
titanic = sns.load_dataset('titanic')

# Calculate mean and standard deviation for 'age' and 'fare'
age_mean = titanic['age'].mean()
age_std = titanic['age'].std()
fare_mean = titanic['fare'].mean()
fare_std = titanic['fare'].std()

# Standard Scaling
titanic['age_standard'] = (titanic['age'] - age_mean) / age_std
titanic['fare_standard'] = (titanic['fare'] - fare_mean) / fare_std

print(titanic[['age', 'age_standard', 'fare', 'fare_standard']].head())

Output:

    age  age_standard     fare  fare_standard
0  22.0     -0.530005   7.2500      -0.502163
1  38.0      0.571430  71.2833       0.786404
2  26.0     -0.254646   7.9250      -0.488580
3  35.0      0.364911  53.1000       0.420494
4  35.0      0.364911   8.0500      -0.486064

Note: The Titanic age column contains missing values. Pandas’ mean() and std() skip missing values by default, so the scaling parameters are calculated only from non-missing ages. Missing ages will remain NaN after scaling. Also, Series.std() uses the sample standard deviation by default, equivalent to ddof=1, which can differ slightly from tools like scikit-learn’s StandardScaler, which uses population standard deviation.