Polynomial Regression

Polynomial Regression | CodeSignal Learn

Lesson Introduction

Hello! Today, we're diving into Polynomial Regression, an advanced form of regression analysis for modeling complex relationships between variables. We'll learn how to use Python and Scikit-Learn to perform polynomial regression. By the end, you'll know how to create polynomial features, train a model, and make predictions. Polynomial regression is useful for capturing non-linear relationships. For instance, predicting exam scores (the target) based on study hours (the feature) might not follow a simple linear pattern. Polynomial regression can help in such cases.

Understanding Polynomial Features

x^2

Loading and Preparing Data

y = 3x^2 + 2x + \text{noise}

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Splitting Data into Training and Test Sets

As always, we'll split our data into training and test sets to train and evaluate our model. We will use X_train to train the model and X_test to evaluate its peformance. Python from sklearn.model_selection import train_test_split # Split the data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Display the shapes of the train/test sets print("X_train shape:", X_train.shape, "y_train shape:", y_train.shape) print("X_test shape:", X_test.shape, "y_test shape:", y_test.shape) # Output: # X_train shape: (80, 1) y_train shape: (80, 1) # X_test shape: (20, 1) y_test shape: (20, 1) from sklearn.model_selection import train_test_split # Split the data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Display the shapes of the train/test sets print("X_train shape:", X_train.shape, "y_train shape:", y_train.shape) print("X_test shape:", X_test.shape, "y_test shape:", y_test.shape) # Output: # X_train shape: (80, 1) y_train shape: (80, 1) # X_test shape: (20, 1) y_test shape: (20, 1)

Training a Simple Linear Regression Model

First, we'll train a simple linear regression model without polynomial features, like we did in the first lesson. Pythonfrom sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error # Train a simple linear regression model linear_model = LinearRegression() linear_model.fit(X_train, y_train) # Make predictions y_pred_linear = linear_model.predict(X_test) # Calculate the mean squared error mse_linear = mean_squared_error(y_test, y_pred_linear) print(f"Linear Regression MSE: {mse_linear}") # Output # Linear Regression MSE: 0.7138921735032644from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error # Train a simple linear regression model linear_model = LinearRegression() linear_model.fit(X_train, y_train) # Make predictions y_pred_linear = linear_model.predict(X_test) # Calculate the mean squared error mse_linear = mean_squared_error(y_test, y_pred_linear) print(f"Linear Regression MSE: {mse_linear}") # Output # Linear Regression MSE: 0.7138921735032644 Now, we have the MSE score for a regular linear regression model. There is not much to say about it, but we can use it to compare this model to others. Let's train a smarter polynomial regression model and check if it works better.

Transforming Features and Training a Polynomial Regression Model

Next, we'll transform the input data to include polynomial terms and train a polynomial regression model. Pythonfrom sklearn.preprocessing import PolynomialFeatures # Transforming the features into polynomial features poly_features = PolynomialFeatures(degree=2) X_train_poly = poly_features.fit_transform(X_train) X_test_poly = poly_features.transform(X_test) # Training a polynomial regression model poly_model = LinearRegression() poly_model.fit(X_train_poly, y_train) # Make predictions y_pred_poly = poly_model.predict(X_test_poly) # Calculate the mean squared error mse_poly = mean_squared_error(y_test, y_pred_poly) print(f"Polynomial Regression MSE: {mse_poly}") # Output # Polynomial Regression MSE: 0.006358406072820809from sklearn.preprocessing import PolynomialFeatures # Transforming the features into polynomial features poly_features = PolynomialFeatures(degree=2) X_train_poly = poly_features.fit_transform(X_train) X_test_poly = poly_features.transform(X_test) # Training a polynomial regression model poly_model = LinearRegression() poly_model.fit(X_train_poly, y_train) # Make predictions y_pred_poly = poly_model.predict(X_test_poly) # Calculate the mean squared error mse_poly = mean_squared_error(y_test, y_pred_poly) print(f"Polynomial Regression MSE: {mse_poly}") # Output # Polynomial Regression MSE: 0.006358406072820809 By applying PolynominalFeatures(degree=2)().fit_transform() to our data (both X_train and X_test), we create a new feature that models a quadratic relationship. Having trained both models, we can now compare their performance using the mean squared error (MSE). Python# Linear Regression MSE: 0.7138921735032644 # Polynomial Regression MSE: 0.006358406072820809# Linear Regression MSE: 0.7138921735032644 # Polynomial Regression MSE: 0.006358406072820809 The polynomial regression model has a much lower MSE, indicating it fits the data much better.

Polynomial Regression

Lesson Introduction

Understanding Polynomial Features

Loading and Preparing Data

Join the 1M+ learners on CodeSignal

Splitting Data into Training and Test Sets

Training a Simple Linear Regression Model

Transforming Features and Training a Polynomial Regression Model

Lesson Summary