Implementing Bag-of-Words Representation

In the world of text analysis, transforming raw data into a format that is both computer-friendly and preserves the essential information for further processing is crucial. One of the simplest yet versatile methods to do this is the Bag-of-Words Representation, or BoW for short.

BoW is essentially a method to extract features from text. Imagine you have a big bag filled with words. These words can come from anywhere: a book, a website, or, in our case, movie reviews from the IMDB dataset. For each document or sentence, the BoW representation will contain the count of how many times each word appears. Most importantly, in this "bag," we don't care about the order of words, only their occurrence.

Consider this simple example with three sentences:

1. The cat sat on the mat.
2. The cat sat near the mat.
3. The cat played with a ball.

Using a BoW representation, our table would look like this:

Each sentence (document) corresponds to a row, and each unique word is a column. The values in the cells represent the word count in the given sentence.

We can start practising the Bag-of-Words model by using Scikit-learn CountVectorizer on the exact same three sentences:

from sklearn.feature_extraction.text import CountVectorizer

# Simple example sentences
sentences = ['The cat sat on the mat.',
             'The cat sat near the mat.',
             'The cat played with a ball.']

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(sentences)

print('Feature names:')
print(vectorizer.get_feature_names_out())
print('Bag of Words Representation:')
print(X.toarray())

The output of the above code will be:

Feature names:
['ball' 'cat' 'mat' 'near' 'on' 'played' 'sat' 'the' 'with']
Bag of Words Representation:
[[0 1 1 0 1 0 1 2 0]
 [0 1 1 1 0 0 1 2 0]
 [1 1 0 0 0 1 0 1 1]]

From the output, you'll notice that Scikit-learn CountVectorizer has done the exact thing as our previous manual process. It's created a Bag-of-Words representation for our sentences where each row corresponds to a sentence and each column to a unique word.