Linear Regression: From Basics to Predictive Modeling
Introduction
Welcome to this fascinating lesson on regression analysis, where we will delve into the realm of Regression. Before we continue working with the California Housing Dataset, we're going to take a brief detour to explain regression with a simpler dataset. This will help us to understand the principles of linear regression, construct a linear regression model in Python, compute coefficients, and predict values with our mathematical model in a more controlled and comprehendible setting. Are you ready to decode regression analysis?
Creating a Simple Dataset
Before implementing our regression model, let's create a simple dataset to be used in our computations. Consider a simple scenario where x represents some feature values (independent variables) and y corresponds to target values (dependent variables). Our aim is to compute the values of y based on x values, therefore finding a line that fits our data.
Let's delve into Python code to shape our dataset:
Python
# We will first define our hypothetical datasetx = [1, 2, 3, 4, 5] # The feature values (independent variable)y = [1, 2, 3, 2, 3] # The target values (dependent variable)
Understanding Regression
Calculation of Regression Line Coefficients
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal
At the heart of statistics lies Regression Analysis, a powerful tool that draws connections among variables. Imagine sketching a line or crafting a curve that best fits the distribution of data on a two-dimensional plane. Pretty cool, right?
With independent variables affecting dependent variables, regression analysis carves out a path to render these connections visible. Look out for Linear Regression, an integral method to predict a dependent variable value (y) based on the value of an independent variable (x). As represented by the following formula:
y=βx+α
Where:
y is the dependent variable we aim to predict.
β represents the slope of the regression line, indicating how much y changes with a unit change in x.
α is the y-intercept of the regression line.
x is the independent variable.
This equation is the cornerstone of linear regression, providing a straightforward approach to predicting the dependent variable based on the independent variable.
In this section, we'll dive into computing the coefficients alpha (α) and beta (β) of our regression line, which are crucial for creating our predictive model. To accomplish this, let's first understand the formulas used to determine α and β.
The slope of the line (β) can be calculated using the formula:
β=∑(xi−xˉ)2∑(xi−xˉ)(yi−yˉ)
Where:
xi and yi are the individual sample points for the independent and dependent variables, respectively.
xˉ and yˉ are the mean values of the independent variable (x) and dependent variable (y), respectively.
Once we have β, we can find the intercept (α) using the formula:
α=yˉ−βxˉ
Now let's transition to the Python code necessary to compute these coefficients.
# Compute the mean (average) of x and ymean_x = sum(x) / len(x)mean_y = sum(y) / len(y)# Calculate the numerator and the denominator for the slope (β)numerator = sum((xi - mean_x) * (yi - mean_y) for xi, yi in zip(x, y))denominator = sum((xi - mean_x) ** 2 for xi in x)# Compute the coefficientsbeta = numerator / denominatoralpha = mean_y - (beta * mean_x)# Printing Coefficientsprint('Beta:', beta ,'Alpha:', alpha) # Output: Beta: 0.4 Alpha: 1.0
First, the numerator is calculated by multiplying the deviations of each x and y value from their means and summing these products, which measures the covariance of x and y and provides insight into how y varies with x. Next, the denominator is determined by summing the squared deviations of x values from their mean, capturing the variance of x and reflecting its spread. Then, β is found by dividing the numerator by the denominator, representing the slope of the regression line and indicating the average change in y per unit change in x. Finally, α is computed by using the mean of y and subtracting the product of β and the mean of x, which appropriately positions the regression line by setting its y-intercept.
Through this streamlined process, we've calculated the essential coefficients that define the relationship between our variables in the linear regression model. Resulting in the formula for our linear regression, which is y=0.4x+1
Python
# Compute the mean (average) of x and ymean_x = sum(x) / len(x)mean_y = sum(y) / len(y)# Calculate the numerator and the denominator for the slope (β)numerator = sum((xi - mean_x) * (yi - mean_y) for xi, yi in zip(x, y))denominator = sum((xi - mean_x) ** 2 for xi in x)# Compute the coefficientsbeta = numerator / denominatoralpha = mean_y - (beta * mean_x)# Printing Coefficientsprint('Beta:', beta ,'Alpha:', alpha) # Output: Beta: 0.4 Alpha: 1.0
Implementing the Regression Model
Armed with alpha and beta, we can now code a function to calculate our regression line.
# Function to make predictionsdef predict_y(alpha, beta, x_i): return beta * x_i + alpha
Making Predictions
It's time to put our regression model to work and conjure up some predictions!
# Making predictionsy_pred = [predict_y(alpha, beta, x_i) for x_i in x]
Visualizing the Data
Lesson Summary and Practice
Congratulations on successfully deciphering regression analysis! We've unraveled significant insights, implemented a linear regression model, visualized predictions, and evaluated the model. Now, let's reinforce your learning with practice exercises. Go ahead and explore the fascinating world of regression analysis!
# Function to make predictionsdef predict_y(alpha, beta, x_i): return beta * x_i + alpha
Python
# Making predictionsy_pred = [predict_y(alpha, beta, x_i) for x_i in x]
Let's illustrate our actual data points and the regression line for a graphical treat, and also include the prediction of a single point based on input and plot it alongside for a comprehensive visualization.
Python
import matplotlib.pyplot as plt# Visualizing the dataplt.scatter(x, y, color='blue') # Actual data pointsplt.plot(x, y_pred, color='red') # Predicted regression line# Predicting and plotting a single data pointx_new = 3.5 # Example new data pointy_new_pred = predict_y(alpha, beta, x_new) # Prediction for the new data pointplt.scatter(x_new, y_new_pred, color='green', s=100, zorder=5) # Plotting the new predicted pointplt.xlabel('X')plt.ylabel('Y')plt.grid(True)plt.show()
In this example, we have not only visualized the actual data points against the predicted regression line but also incorporated a step to predict and plot a single data point based on an input (x_new). The green dot represents this new predicted value, distinctively highlighted on the plot to easily discern it from the existing data. This addition vividly demonstrates how new predictions can be made and visualized within the context of the original dataset and regression analysis. Just as we would plug the new value of 3.5 to our formula and predict like this: 2.4=0.4×3.5+1
