Understanding and Implementing Simple Linear Regression from Scratch

Introduction

Hey there, are you ready to dive into another exciting journey? Today's adventure spot is Regression Analysis, a powerful tool for modeling relationships between variables. Our quest will be to explore the realm of Simple Linear Regression and implement it from scratch using Python! Imagine being able to predict future outcomes based on specific parameters — exciting, isn't it? That's the magic of Regression Analysis! Let's gear up and embark on this journey with Simple Linear Regression, a storyline featuring two main characters: dependent and independent variables.

Understanding Regression

Regression, a superstar in the world of statistics, finance, investing, and Machine Learning, is our guide to predicting future outcomes. In the grand world of regression, there are two central provinces: Simple Linear Regression and Multiple Linear Regression. Our map is marked to travel through the province of Simple Linear Regression. To get you excited, let's take an example. Suppose you own a restaurant and want to predict your sales for the next week. You pull out your past data, relating your advertisement hours with sales details. Fancy predicting sales using advertisement data? Buckle up, as that's where we are heading!

Basics of Simple Linear Regression

y = c + m*x

Mathematical Basis of Simple Linear Regression

m = \frac{\sum_{i=1}^N{(x_i - \overline{x})(y_i - \overline{y})}}{\sum_{i=1}^N{(x_i - \overline{x})^2}}

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Python

import numpy as np

# Step 1: Get the data set
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 5])

# Step 2: Compute the mean of the X and y
mean_x = np.mean(x)
mean_y = np.mean(y)

# Step 3: Calculate the coefficients
m = np.sum((x - mean_x) * (y - mean_y)) / np.sum((x - mean_x)**2)
c = mean_y - m * mean_x

# Voila! We have our model
print(f"Model: y = {c} + {m}*x")  # Output: Model: y= 2.2 + 0.6*x

import numpy as np

# Step 1: Get the data set
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 5])

# Step 2: Compute the mean of the X and y
mean_x = np.mean(x)
mean_y = np.mean(y)

# Step 3: Calculate the coefficients
m = np.sum((x - mean_x) * (y - mean_y)) / np.sum((x - mean_x)**2)
c = mean_y - m * mean_x

# Voila! We have our model
print(f"Model: y = {c} + {m}*x")  # Output: Model: y= 2.2 + 0.6*x

Python

plt.scatter(x, y, color="red", marker="o", s=30)  # s is marker size
y_pred = c + m * x
plt.plot(x, y_pred, color="blue")
plt.xlabel('x')
plt.ylabel('y')
plt.title('Simple Linear Regression')
plt.show()

plt.scatter(x, y, color="red", marker="o", s=30)  # s is marker size
y_pred = c + m * x
plt.plot(x, y_pred, color="blue")
plt.xlabel('x')
plt.ylabel('y')
plt.title('Simple Linear Regression')
plt.show()

Python

import numpy as np

# Step 1: Get the data set
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 5])

# Step 2: Compute the mean of the X and y
mean_x = np.mean(x)
mean_y = np.mean(y)

# Step 3: Calculate the coefficients
m = np.sum((x - mean_x) * (y - mean_y)) / np.sum((x - mean_x)**2)
c = mean_y - m * mean_x

# Voila! We have our model
print(f"Model: y = {c} + {m}*x")  # Output: Model: y= 2.2 + 0.6*x

Python

plt.scatter(x, y, color="red", marker="o", s=30)  # s is marker size
y_pred = c + m * x
plt.plot(x, y_pred, color="blue")
plt.xlabel('x')
plt.ylabel('y')
plt.title('Simple Linear Regression')
plt.show()

Introduction

Understanding Regression

Basics of Simple Linear Regression

Mathematical Basis of Simple Linear Regression

Join the 1M+ learners on CodeSignal

Implementing Simple Linear Regression from Scratch

Visualizing Linear Regression

Lesson Summary and Practice