Understanding PCA Foundations

Embark on an exciting journey through the world of Principal Component Analysis (PCA). In this lesson, we will explore the indispensable roles of eigenvalues and eigenvectors in understanding the PCA framework and dive into the computation of these mathematical constructs using R. Our adventure will cover the essential role of the covariance matrix and how to compute it. Ready? Set? Let's start!

At the onset, we start with a dataset housing different physical measures: weight_lbs (in lbs), height_inches (in inches), and height_cm (in cm). We capture these in R using vectors and combine them into a data frame for easy manipulation:

# Given data
weight_lbs <- c(150, 160, 155, 165, 170, 160, 158, 175, 180, 170)
height_inches <- c(68, 72, 66, 69, 71, 65, 67, 70, 73, 68)
height_cm <- c(172.72, 182.88, 167.64, 175.26, 180.34, 165.1, 170.18, 177.8, 185.42, 172.72)

# Create a data frame
df <- data.frame(
  weight_lbs = weight_lbs,
  height_inches = height_inches,
  height_cm = height_cm
)

Here, the data frame df represents our collected dataset.

Before performing Principal Component Analysis, we need to standardize the data. This means changing the scale of our data so that each feature has a mean of 0 and a standard deviation of 1.

PCA is sensitive to the scale of the features. Features with larger scales will dominate the variance calculations and may bias the results toward those features. Standardizing the data ensures that each feature contributes equally to the analysis, preventing this bias.

To standardize the data, each feature is transformed using the following formula:

$X_{standardized} = \frac{X - \mu}{\sigma}$

Where:

• $X_{standardized}$ is the standardized value of the feature.
• $X$ is the original value of the feature.
• $\mu$ is the mean of the feature.
• $\sigma$ is the standard deviation of the feature.

Let's standardize just the two height columns in our dataset using R's scale() function and visualize the standardized data using ggplot2:

library(ggplot2)

# Select the height columns and standardize
X <- df[, c("height_inches", "height_cm")]
X_standard <- scale(X)

# Convert to data frame for plotting
X_standard_df <- as.data.frame(X_standard)
colnames(X_standard_df) <- c("height_inches", "height_cm")

# Plot standardized data
ggplot(X_standard_df, aes(x = height_inches, y = height_cm)) +
  geom_point(color = "blue") +
  ggtitle("Standardized Data") +
  xlab("height_inches") +
  ylab("height_cm") +
  theme_bw() +
  theme(panel.grid = element_line())

Output:

After standardization, our data is now centered and scaled, making the variables more comparable.