Principal Component Analysis in R

Welcome to this lesson on Principal Component Analysis (PCA), a powerful technique widely applied in data analysis and machine learning to reduce high-dimensional data into lower dimensions, effectively simplifying the dataset while still retaining the relevant information. In this lesson, we'll look at how we can prepare our data, how to apply PCA using R, how to understand the percentage of variance explained by each principal component (explained variance ratio), and finally, how to visualize the results of our PCA.

Before moving forward, let's first apply what we've learned to a dataset to standardize the data:

# Define the dataset
data <- data.frame(
  weight_lbs = c(150, 160, 155, 165, 170, 160, 158, 175, 180, 170),
  height_inches = c(68, 72, 66, 69, 71, 65, 67, 70, 73, 68),
  height_cm = c(172.72, 182.88, 167.64, 175.26, 180.34, 165.1, 170.18, 177.8, 185.42, 172.72)
)

# Standardize the data
data_scaled <- as.data.frame(scale(data))

Next, we apply PCA, a technique that first computes the covariance matrix of the data, followed by finding its eigenvectors and eigenvalues. The eigenvectors corresponding to the $n$ largest eigenvalues are then used to project the data into an $n$-dimensional subspace.

Let's perform PCA and explore the key outputs from the prcomp() function:

# Perform PCA
pca_result <- prcomp(data_scaled, center = FALSE, scale. = FALSE)

The main components of the prcomp output are:

• pca_result$sdev: The standard deviations of the principal components (i.e., the square roots of the eigenvalues).
• pca_result$rotation: The matrix of variable loadings (eigenvectors), showing how each original variable contributes to each principal component.
• pca_result$x: The principal component scores, i.e., the coordinates of the data in the new principal component space.
• pca_result$center and pca_result$scale: The centering and scaling applied to the original data (here, both are NULL because we already scaled the data).

Let's inspect these outputs:

# Standard deviations of principal components
print(pca_result$sdev)

# Loadings (eigenvectors)
print(pca_result$rotation)

# Center and scale used (should be NULL since we pre-scaled)
print(pca_result$center)
print(pca_result$scale)

# Project the data onto the first two principal components (scores)
pca_scores <- pca_result$x[, 1:2]

• pca_result$sdev tells you how much variance each principal component captures (higher values mean more variance).
• pca_result$rotation shows the weights (loadings) for each original variable in each principal component.
• pca_result$x contains the transformed data in the new coordinate system.