Exploring Linear Discriminant Analysis with Scikit-Learn

Welcome back to our journey through "Linear Landscapes of Dimensionality Reduction". Today's lesson focuses on applying Linear Discriminant Analysis (LDA) using Scikit-learn and then diving into feature extraction and selection. Primarily, we'll be utilizing the LinearDiscriminantAnalysis function from sklearn.discriminant_analysis. Ready to take the plunge?

Before diving in, let's do a quick revision of LDA, how it works, and its implementation. As we transition to using Scikit-learn today, our first step is to load the dataset we will be working with. We'll load the Iris dataset, which is included in Scikit-learn's datasets:

from sklearn.datasets import load_iris

# Load the Iris dataset
data = load_iris()
X = data.data

Here, data is a bunch object containing both data and targets (species) among other attributes, while X captures the features data.

Our next step is to preprocess our data. We scale the features to a zero mean and unit variance, important for optimal LDA performance:

import numpy as np

# Scale the features to zero mean and unit variance
X = (X - np.mean(X, axis=0)) / np.std(X, axis=0)

This code subtracts the mean and divides by the standard deviation for each feature column, effectively standardizing it.

Next, we'll split our dataset into training and testing data:

X_train = X[:120]
y_train = data.target[:120]

X_test = X[120:]
y_test = data.target[120:]

X_train and y_train make up our training set, while X_test and y_test are our testing sets.

Let's apply LDA to our data:

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

lda = LinearDiscriminantAnalysis(n_components=2)
X_lda = lda.fit_transform(X_train, y_train)

We initiate the Linear Discriminant Analysis object with 2 components (our target), fit the model to our training data, and transform it into a lower-dimensional space.