Meeting Polynomial Regression

Meeting Polynomial Regression | CodeSignal Learn

Introduction

Hey there! Welcome to another enlightening session on predictive modeling where we're diving into Regression Models, specifically Polynomial Regression, using Python along with the sklearn library. Think of Polynomial Regression as an extended version of Linear Regression, which is capable of modeling the relationship between two variables, i.e., predictors (x) and response (y), as an nth degree polynomial. By the end of this lesson, the main goal is to own the practical knowledge of how Polynomial Regression works and how to implement the same in Python using sklearn. Let's start!

The Concept: Polynomial Regression

First and foremost, let's try to understand what Polynomial Regression really entails. At its core, Polynomial Regression extends the simple linear regression by adding extra predictors, which are derived by raising each of the original predictors to a power. This extension enables us to encapsulate relationships between the variable that are not merely linear. Suppose you're trying to estimate the price of a house. While the price depends on its size, the correlation isn't linear because the price does not increase proportionally with the size. This is where Polynomial Regression comes in! However, one must be cautious. Use of a very high degree polynomial can lead to complex models which might result in overfitting.

Understanding the Mathematics

y = b_0 + b_1x + b_2x^2

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

In Polynomial Regression, the relationship between the independent variable x and the dependent variable y is modeled as an nth degree polynomial. This intricate process begins by elevating the basic premise of simple linear regression. We add extra predictors derived by raising each of the original predictors to a power, thus introducing the concept of polynomial degree. Herein, the degree signifies the highest power of any predictor in the model, distinguishing it fundamentally from the number of features. While the degree describes the highest exponent, the number of features pertains to the distinct variables in the dataset.

Consider the equation for a single predictor variable model: $y = b_0 + b_1x + b_2x^2$

$y$ targets the variable we aim to predict,
$x$ is the predictor,
$b_0$ is the y-intercept,
$b_1x$ represents the linear term,
$b_2x^2$ contributes the quadratic term, adding curvature and thus, a nonlinear aspect.

This equation demonstrates a polynomial of 2nd degree; however, despite having a single feature (x), we integrate its power to create multiple predictors ( $x$ , $x^2$ ). In scenarios with multiple features, each feature can be similarly elevated to create a richer set of predictors. Consequently, while the model complexity increases with the degree due to the heightened curvature, it's crucial to distinguish this complexity from the actual number of features within the dataset.

Through the adaptive nature of Polynomial Regression, we can decode complex, nonlinear relationships by meticulously managing the balance between the polynomial degree and the dataset's intrinsic dimensionality. This balance is paramount in unveiling the model's predictive prowess while avoiding the pitfalls of overfitting, thus unleashing the full potential of Polynomial Regression in capturing a wide spectrum of data nuances.

Data Preparation

Let's walk through preparing our data for implementing Polynomial Regression. We create a simple dataset, then reshape it for compatibility with sklearn. The simplification to a handmade dataset allows us to avert the complexity of real-world data, focusing purely on the mechanics of Polynomial Regression. Here's how we go about it: Python import numpy as np # Creating our dataset X = np.array([5, 6, 7, 8, 9]) Y = np.array([4, 6, 10, 15, 23]) # Reshaping to 2D for compatibility with sklearn X = X.reshape(-1, 1) import numpy as np # Creating our dataset X = np.array([5, 6, 7, 8, 9]) Y = np.array([4, 6, 10, 15, 23]) # Reshaping to 2D for compatibility with sklearn X = X.reshape(-1, 1) With our data ready and well-prepared, we can traverse the path of implementing and analyzing Polynomial Regression.

Creating Polynomial Features

Implementing Polynomial Regression in sklearn requires us first to transform our input variables into polynomial features. This transformation step is vital as it equips a linear model with the capability to fit more complex relationships by introducing higher-order terms for the independent and dependent variables. Python from sklearn.preprocessing import PolynomialFeatures # Creating polynomial features poly_features = PolynomialFeatures(degree=2) # Transforming the input features X_poly = poly_features.fit_transform(X) # Displaying the transformation print("Original feature:", X[0]) print("Transformed polynomial feature:", X_poly[0]) from sklearn.preprocessing import PolynomialFeatures # Creating polynomial features poly_features = PolynomialFeatures(degree=2) # Transforming the input features X_poly = poly_features.fit_transform(X) # Displaying the transformation print("Original feature:", X[0]) print("Transformed polynomial feature:", X_poly[0]) In the output, observe how the original feature is expanded into its polynomial counterpart, enhancing the model's capability to decipher more complex patterns in the data. text Original feature: [5] Transformed polynomial feature: [ 1. 5. 25.] Original feature: [5] Transformed polynomial feature: [ 1. 5. 25.]

Implementing the Polynomial Regression Model

b_0 = 22.34

Visualizing Polynomial Regression Curve

After developing our Polynomial Regression model, it's insightful to visualize how well the model fits our data. To do so, we generate a smooth curve that represents our Polynomial Regression model's predictions. This step requires creating a dense range of input values (X_smooth), transforming these into polynomial features, and then using our model to predict the corresponding Y values. This approach allows us to plot a smooth curve that closely follows the trends captured by our model, offering a clear visualization of the model's performance with respect to the actual data points. Python import matplotlib.pyplot as plt # Creating more points for a smoother curve X_smooth = np.linspace(X.min(), X.max(), 300).reshape(-1, 1) # Reshape for transformation X_poly_smooth = poly_features.transform(X_smooth) # Transforming for polynomial regression Y_pred_smooth = model.predict(X_poly_smooth) # Predicting the values # Plotting the smoother regression line plt.scatter(X.flatten(), Y, color='blue') # Plotting the actual data points plt.plot(X_smooth, Y_pred_smooth, color='red') plt.title("Polynomial Regression") plt.xlabel('X') plt.ylabel('Y') plt.grid(True) plt.show() import matplotlib.pyplot as plt # Creating more points for a smoother curve X_smooth = np.linspace(X.min(), X.max(), 300).reshape(-1, 1) # Reshape for transformation X_poly_smooth = poly_features.transform(X_smooth) # Transforming for polynomial regression Y_pred_smooth = model.predict(X_poly_smooth) # Predicting the values # Plotting the smoother regression line plt.scatter(X.flatten(), Y, color='blue') # Plotting the actual data points plt.plot(X_smooth, Y_pred_smooth, color='red') plt.title("Polynomial Regression") plt.xlabel('X') plt.ylabel('Y') plt.grid(True) plt.show() This plot vividly illustrates the non-linear relationship captured by our Polynomial Regression model, differing substantially from what we'd expect with Simple Linear Regression.