Random Search in Machine Learning

Choosing the right parameters in machine learning models can greatly affect their success. Imagine these parameters as cake ingredients: the right amount makes your cake delicious. Similarly, the right parameter settings make your model accurate. Random Search helps find these “right ingredients” by trying random combinations. By the end of this lesson, you will:

• Understand what Random Search is
• Learn how to implement it using Scikit-Learn
• Interpret the results to improve models

Random Search is a technique for tuning parameters by randomly sampling combinations from a given range, like randomly picking recipes to see which cake tastes best. Unlike Grid Search, which tries every possible combination, Random Search is faster because it tries random ones. It’s like flipping through a recipe book and picking random recipes instead of trying every single one.

We’ll use the wine dataset from Scikit-Learn. Let's load it and scale features:

from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler

# Load real dataset
X, y = load_wine(return_X_y=True)
X = StandardScaler().fit_transform(X)

To evaluate our model, we split the dataset into a training set (80%) and a testing set (20%).

from sklearn.model_selection import train_test_split

# Splitting the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

A parameter grid is a set of parameters you want to try. For Logistic Regression, we’ll tune C and solver.

# Defining the parameter grid
param_distributions = {
    'C': [0.1, 0.5, 0.75, 1, 5, 10, 25, 50, 75, 100],
    'solver': ['liblinear', 'saga']
}

• C: Controls the strength of regularization. Smaller values specify stronger regularization.
• solver: Algorithm used in the optimization problem.

RandomizedSearchCV is a Scikit-Learn tool for Random Search. It randomly selects parameter combinations and evaluates their performance.

• n_iter: Number of settings sampled.
• cv: Number of cross-validation splits.

from sklearn.model_selection import RandomizedSearchCV
from sklearn.linear_model import LogisticRegression

# Performing randomized search
random_search = RandomizedSearchCV(LogisticRegression(max_iter=1000), param_distributions, n_iter=10, cv=5, random_state=42)
random_search.fit(X_train, y_train)