Univariate Feature Selection

Hello, and welcome to this lesson on univariate statistical tests for feature selection in machine learning using R. The effective management of dataset features can significantly influence the performance of your machine learning models. By carefully selecting the most relevant features, you can improve model accuracy, reduce overfitting, and decrease training time. One widely used approach for this is univariate selection for feature selection. In this lesson, we will explore how to perform univariate feature selection in R, focusing on statistical tests that help identify the most informative features in your dataset. By the end of this session, you will understand how to use univariate feature selection in R and appreciate its strengths and limitations.

Univariate statistical tests evaluate each feature independently to determine its relationship with the response variable. These tests are straightforward to apply and interpret, providing valuable insights into your data. In base R, we can use the chi-square test (chisq.test) to assess association between each feature and the target. Because the iris features are numeric, we first discretize each feature into bins, create a contingency table versus the target, compute the chi-square statistic, and then rank features by that score.

For this tutorial, we will use the built-in iris dataset in R. The iris dataset contains measurements for 150 iris flowers from three different species. It includes five attributes: Sepal.Length, Sepal.Width, Petal.Length, Petal.Width, and Species. The Species column is our target variable, while the other columns are the features.

Here's how you load and inspect the dataset in R:

# Load the iris dataset
data(iris)

# View the structure of the dataset
str(iris)

# Check the dimensions of the dataset
dim(iris) # 150 rows, 5 columns

This output shows that the dataset has 150 samples, each with 4 feature variables and 1 target variable (Species). In R, the iris dataset is stored as a data frame, where each column represents a variable.