Practical Guide to Principal Component Analysis (PCA) in Data Science

Practical Guide to Principal Component Analysis (PCA) in Data Science | CodeSignal Learn

Topic Overview and Actualization

Welcome to our journey into high-dimensional data and the associated challenges that it presents. We'll focus on Principal Component Analysis (PCA), a significant method in the realm of dimensionality reduction. Running through a real-world example, we'll implement PCA in Python. This lesson's roadmap proceeds as follows: Introducing high-dimensional data and understanding its challenges Establishing the need for dimensionality reduction Unveiling PCA, its algorithm, and benefits Implementing PCA using Python

Understanding High-Dimensional Data

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Simple PCA Implementation in Python

Let's define some functions to standardize our data and compute the covariance matrix. Don't worry about the implementations for now. We will cover them in the next lesson. Python import numpy as np def standardize(X): return (X - np.mean(X, axis=0)) / np.std(X, axis=0) def compute_covariance_matrix(X): return np.dot(X.T, X) / (X.shape[0] - 1) import numpy as np def standardize(X): return (X - np.mean(X, axis=0)) / np.std(X, axis=0) def compute_covariance_matrix(X): return np.dot(X.T, X) / (X.shape[0] - 1) Python def PCA(X, num_components): X = standardize(X) covariance_matrix = compute_covariance_matrix(X) eigenvalues, eigenvectors = np.linalg.eig(covariance_matrix) idx = np.argsort(eigenvalues)[::-1] eigenvalues = eigenvalues[idx][:num_components] eigenvectors = eigenvectors[:, idx][:, :num_components] # select first num_components eigenvectors return eigenvalues, eigenvectors def PCA(X, num_components): X = standardize(X) covariance_matrix = compute_covariance_matrix(X) eigenvalues, eigenvectors = np.linalg.eig(covariance_matrix) idx = np.argsort(eigenvalues)[::-1] eigenvalues = eigenvalues[idx][:num_components] eigenvectors = eigenvectors[:, idx][:, :num_components] # select first num_components eigenvectors return eigenvalues, eigenvectors Let's break down each step. First, we standardize the data and compute the covariance matrix. We then obtain the eigenvalues and eigenvectors using the np.linalg.eig function. Then, we sort the eigenvalues, as higher eigenvalues indicate higher variability in our data. We take the num_components highest eigenvalues and their corresponding eigenvectors. These eigenvectors are the principal components. Note that here, the number of components is the number of dimensions we want to reduce the dataset to.

PCA in Action.

Let's use PCA to combine the two height features into a single principal component. Python X = df[['Height (inches)', 'Height (cm)']].to_numpy() eigenvalues, eigenvectors = PCA(X,1) X_pca = np.dot(X, eigenvectors) # Plotting original data plt.figure(figsize=(20, 10)) plt.subplot(1, 2, 1) plt.scatter(X[:, 0], X[:, 1], color='b') plt.xlabel('Height (inches)') plt.ylabel('Height (cm)') plt.title('Original Data') plt.grid(True) # Plotting data after PCA plt.subplot(1, 2, 2) plt.scatter(X_pca[:,0], [0]*len(X_pca), color='r') plt.xlabel('Principal Component 1') plt.title('Data after PCA') plt.grid(True) plt.show() X = df[['Height (inches)', 'Height (cm)']].to_numpy() eigenvalues, eigenvectors = PCA(X,1) X_pca = np.dot(X, eigenvectors) # Plotting original data plt.figure(figsize=(20, 10)) plt.subplot(1, 2, 1) plt.scatter(X[:, 0], X[:, 1], color='b') plt.xlabel('Height (inches)') plt.ylabel('Height (cm)') plt.title('Original Data') plt.grid(True) # Plotting data after PCA plt.subplot(1, 2, 2) plt.scatter(X_pca[:,0], [0]*len(X_pca), color='r') plt.xlabel('Principal Component 1') plt.title('Data after PCA') plt.grid(True) plt.show() We've successfully merged two features into a single critical one while retaining essential information.

Applying Reduction From 3D to 2D

Now that we've seen how to reduce 2-dimensional data to 1-dimensional data let's reduce our original 3-dimensional data to 2 dimensions. Python X = df[['Height (inches)', 'Height (cm)', 'Weight (lbs)']].to_numpy() eigenvalues, eigenvectors = PCA(X, 2) X_pca = np.dot(X, eigenvectors) plt.scatter(X_pca[:,0], X_pca[:,1]) # Setting labels plt.xlabel('Principal Component 1') plt.ylabel('Principal Component 2') plt.title('Scatter Plot of 2 Principal Components') # Show plot plt.grid(True) plt.show() X = df[['Height (inches)', 'Height (cm)', 'Weight (lbs)']].to_numpy() eigenvalues, eigenvectors = PCA(X, 2) X_pca = np.dot(X, eigenvectors) plt.scatter(X_pca[:,0], X_pca[:,1]) # Setting labels plt.xlabel('Principal Component 1') plt.ylabel('Principal Component 2') plt.title('Scatter Plot of 2 Principal Components') # Show plot plt.grid(True) plt.show() The plot successfully shows how to model our 3-dimensional data into two principal components.

Topic Overview and Actualization

Understanding High-Dimensional Data

Join the 1M+ learners on CodeSignal

Challenges of High-Dimensional Data

Introduction to PCA Algorithm

Simple PCA Implementation in Python

PCA in Action.

Applying Reduction From 3D to 2D

Lesson Summary and Practice