Grid Search: Finding Optimal Model Parameters

Welcome! Today, we're going to learn about an exciting and powerful tool in machine learning called Grid Search. Imagine trying to find the perfect pair of shoes that fit just right. Grid Search does something similar but for tuning machine learning models. By the end of this lesson, you'll understand how to use Grid Search to find the best settings (parameters) for your models.

Imagine baking the perfect cake. You need to find the right proportions of sugar, flour, and baking soda. Grid Search does the same for machine learning models by trying different combinations of parameters to find the best one. Parameters are settings you can adjust to improve your model's performance. The right parameters can make your model more accurate.

The parameters we set when initializing the model are called hyperparameters. Finding the perfect combination of them is called hypertuning.

We have already done some hypertuning before in this course path using for loops. But writing a for loop each time can be laborious, especially if you must check multiple models with multiple hyperparameters each. So, it is time for us to learn about a special tool that automates this process!

Let's implement Grid Search using Scikit-Learn.

First, load the libraries and the Wine dataset:

from sklearn.model_selection import GridSearchCV
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split

# Load real dataset
X, y = load_wine(return_X_y=True)
# Splitting the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Here, we're loading a dataset about different wine types. It contains data to help us classify different wine categories based on their attributes. We also split our data into a training set and a test set.

Next, let's define which parameters to test, similar to adjusting ingredients in a recipe. For DecisionTreeClassifier, try different values of max_depth (the maximum depth of the tree) and min_samples_split (the minimum number of samples required to split an internal node).

The Grid Search requires a parameter grid, defined as a dictionary, where keys are the model's hyperparameters, and the values are the lists of possible options. Let's define it:

# Defining the parameter grid
param_grid = {
    'max_depth': [3, 5, 7, 10],
    'min_samples_split': [2, 5, 10]
}

Here, we say that max_depth can be 3, 5, 7, or 10, and min_samples_split can be 2, 5, or 10.