Naive Bayes Basics

Naive Bayes Basics | CodeSignal Learn

Lesson Introduction

Hey there! Today we are going to explore an exciting topic in machine learning called Naive Bayes . By the end of this lesson, you'll understand what Naive Bayes is and how to implement it using Python's Scikit-Learn library. Let’s dive in!

Understanding Naive Bayes

P(C|X) = \frac{P(X|C) \cdot P(C)}{P(X)}

How Naive Bayes Learns

Naive Bayes updates its likelihoods and priors using the training data. When the model encounters new data, it breaks the data into its constituent features and applies Bayes' Theorem to calculate the class probabilities. The class with the highest probability is the predicted class. We will focus on GaussianNB, commonly used when features are continuous and assumed to follow a normal (Gaussian) distribution.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Naive Bayes is a classification algorithm based on Bayes' Theorem. Imagine you’re a detective using clues (features) to decide who the culprit is (class). Naive Bayes helps by calculating probabilities.

Bayes' Theorem is stated as:

$P(C|X) = \frac{P(X|C) \cdot P(C)}{P(X)}$

Where:

$P(C|X)$ is the posterior probability of class $C$ given predictor $X$ .
$P(X|C)$ is the likelihood which is the probability of predictor $X$ given class $C$ .
$P(C)$ is the prior probability of class $C$ .
$P(X)$ is the prior probability of predictor $X$ .

How Naive Bayes Works

Prior Probability: The algorithm starts by calculating the prior probability for each class based on the training data. It is simply the probability of a sample being of the class $C$ if we know no data about the sample. For example, imagine we predict if the email is spam or not. If the 93% of the emails in the data are not spam, then it is reasonable to suppose that a given email will be not spam with the probability of 93%. This is what the prior probability is.
Likelihood: For each feature, the likelihood (probability of the feature given the class) is calculated. It is essentially the probability of a sample with a given feature to be of the given class.
Independent Features Assumption (Naive Assumption): Assumes that the features are independent, which simplifies calculations.
Posterior Probability: Using Bayes' Theorem, the posterior probability of each class is computed given the feature values. The class with the highest posterior probability is chosen as the prediction.

Before training our Naive Bayes classifier, we need data. Consider it like needing mystery stories before solving them! We'll use the Iris dataset, which includes features of iris flowers to classify them into species. Let’s quickly remind ourselves how to load the dataset using Scikit-Learn : Python from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split # Load the Iris dataset X, y = load_iris(return_X_y=True) # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42) from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split # Load the Iris dataset X, y = load_iris(return_X_y=True) # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)

Training the Naive Bayes Classifier

Now that we have our data split, it’s time to train our Naive Bayes classifier using GaussianNB : Python from sklearn.naive_bayes import GaussianNB # Initialize the Naive Bayes classifier nb_clf = GaussianNB() # Train the classifier with training data nb_clf.fit(X_train, y_train) from sklearn.naive_bayes import GaussianNB # Initialize the Naive Bayes classifier nb_clf = GaussianNB() # Train the classifier with training data nb_clf.fit(X_train, y_train) Here, fit trains the model using the training data, much like a student learning from textbooks.

Making Predictions and Calculating Accuracy

After training the model, let’s make predictions on the test data and calculate the accuracy: Pythonfrom sklearn.metrics import accuracy_score # Make predictions on the testing set y_pred = nb_clf.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Bayes model accuracy: {accuracy * 100:.2f}%") # Bayes model accuracy: 96.67%from sklearn.metrics import accuracy_score # Make predictions on the testing set y_pred = nb_clf.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Bayes model accuracy: {accuracy * 100:.2f}%") # Bayes model accuracy: 96.67% Here, y_pred contains the predicted class labels for the test set, and accuracy_score compares these predictions to the true labels (y_test) to calculate the model's accuracy.