Training a Naive Bayes Classifier for Text Categorization

Hello again! Now that you are familiar with loading and preprocessing the dataset, today we'll learn how to train a Naive Bayes Classifier for text classification using Python. We'll be using the SMS Spam collection dataset in this example given below. By the end of this lesson, you will have a clear understanding about the principles of Naive Bayes algorithm and its implementation with the sklearn package in Python.

You have previously learned how to load our dataset and vectorize its messages, now it's time to split the data in two sets. The train set, where we already know the output that the model learns on, and the test set, where we test our model’s predictions on unseen data. We can leverage the sklearn package for that as done in the following code:

from sklearn.model_selection import train_test_split

# Split data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X_tfidf, df['label'], test_size=0.25, random_state=42)

The random state in our train-test split function ensures reproducibility. This is crucial so we invariably get the same split each time we run the code. Let's use a test size of 0.25, which means we will use 75% data for training and 25% for testing our model.

The Naive Bayes classifier is a simple yet effective and commonly-used, probabilistic classifier. It's founded on applying Bayes' theorem with strong (naive) independence assumptions between the features. Naive Bayes classifiers have been particularly effective for high-dimensional data, and have worked quite well for text classification problems.

In our case, we'll use the Multinomial Naive Bayes implementation available in sklearn, which is suitable for classification with discrete features (like word counts for text classification).

Let's go ahead and create a Multinomial Naive Bayes model. Then, we can train it (or "fit" it, as we say in Machine Learning) with fit(X_train, y_train). Here X_train contains the vectorized training data and y_train contains the corresponding labels.

from sklearn.naive_bayes import MultinomialNB

# Train a Multinomial Naive Bayes classifier
clf = MultinomialNB()
clf.fit(X_train, y_train)