Principal Component Analysis in R

Principal Component Analysis in R | CodeSignal Learn

Introduction

Welcome to this lesson on Principal Component Analysis (PCA), a powerful technique widely applied in data analysis and machine learning to reduce high-dimensional data into lower dimensions, effectively simplifying the dataset while still retaining the relevant information. In this lesson, we'll look at how we can prepare our data, how to apply PCA using R, how to understand the percentage of variance explained by each principal component (explained variance ratio), and finally, how to visualize the results of our PCA.

Preparing the Data

Before moving forward, let's first apply what we've learned to a dataset to standardize the data: R # Define the dataset data <- data.frame(weight_lbs = c(150, 160, 155, 165, 170, 160, 158, 175, 180, 170), height_inches = c(68, 72, 66, 69, 71, 65, 67, 70, 73, 68), height_cm = c(172.72, 182.88, 167.64, 175.26, 180.34, 165.1, 170.18, 177.8, 185.42, 172.72)) # Standardize the data data_scaled <- as.data.frame(scale(data))

PCA with R

n

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Next, we apply PCA, a technique that first computes the covariance matrix of the data, followed by finding its eigenvectors and eigenvalues. The eigenvectors corresponding to the $n$ largest eigenvalues are then used to project the data into an $n$ -dimensional subspace.

Let's perform PCA and explore the key outputs from the prcomp() function:

# Perform PCA
pca_result <- prcomp(data_scaled, center = FALSE, scale. = FALSE)

The main components of the prcomp output are:

pca_result$sdev: The standard deviations of the principal components (i.e., the square roots of the eigenvalues).
pca_result$rotation: The matrix of variable loadings (eigenvectors), showing how each original variable contributes to each principal component.
pca_result$x: The principal component scores, i.e., the coordinates of the data in the new principal component space.
pca_result$center and pca_result$scale: The centering and scaling applied to the original data (here, both are NULL because we already scaled the data).

Let's inspect these outputs:

# Standard deviations of principal components
print(pca_result$sdev)

# Loadings (eigenvectors)
print(pca_result$rotation)

# Center and scale used (should be NULL since we pre-scaled)
print(pca_result$center)
print(pca_result$scale)

# Project the data onto the first two principal components (scores)
pca_scores <- pca_result$x[, 1:2]

pca_result$sdev tells you how much variance each principal component captures (higher values mean more variance).
pca_result$rotation shows the weights (loadings) for each original variable in each principal component.
pca_result$x contains the transformed data in the new coordinate system.

The Importance of Explained Variance Ratio

An informative aspect of PCA is the explained variance ratio, which signals the proportion of the data's variance falling along the direction of each principal component. This information is key, as it tells us how much information we would lose if we ignored the less important dimensions and kept only the ones contributing most to the variance. You can view the explained variance ratio using the summary() function: R# View explained variance ratio summary_pca <- summary(pca_result) print(summary_pca$importance)# View explained variance ratio summary_pca <- summary(pca_result) print(summary_pca$importance) The importance matrix has three rows: Standard deviation: The standard deviation of each principal component (pca_result$sdev). Proportion of Variance: The fraction of the total variance explained by each principal component. Cumulative Proportion: The cumulative variance explained up to each principal component. To extract the proportion of variance explained by the first two principal components: Rexplained_variance <- summary_pca$importance[2, 1:2] print(explained_variance)explained_variance <- summary_pca$importance[2, 1:2] print(explained_variance) The output will be similar to: PC1 PC2 0.8400996 0.1599004 PC1 PC2 0.8400996 0.1599004 This means that the first principal component explains about 84% of the variance, while the second principal component explains about 16% of the variance. In this case, the first principal component is the most important one, as it captures the majority of the variance in the data.

Visualizing the Results of PCA

Finally, let's visualize our PCA results via a scatter plot using ggplot2 . Such a visualization can be incredibly insightful when dealing with high-dimensional data, letting us see the structure of our data in lower-dimensional space. R library(ggplot2) # Create a data frame for plotting plot_data <- as.data.frame(pca_scores) colnames(plot_data) <- c("PC1", "PC2") ggplot(plot_data, aes(x = PC1, y = PC2)) + geom_point(size = 3) + labs(x = "Principal Component 1", y = "Principal Component 2", title = "PCA on Measurement Dataset") + theme_bw() + theme(panel.grid = element_line()) library(ggplot2) # Create a data frame for plotting plot_data <- as.data.frame(pca_scores) colnames(plot_data) <- c("PC1", "PC2") ggplot(plot_data, aes(x = PC1, y = PC2)) + geom_point(size = 3) + labs(x = "Principal Component 1", y = "Principal Component 2", title = "PCA on Measurement Dataset") + theme_bw() + theme(panel.grid = element_line()) Output: You will see a scatter plot showing the data projected onto the first two principal components.