Efficient Text Data Representation with Sparse Matrices

Hello and welcome to this lesson on "Efficient Text Data Representation with Sparse Matrices". As you recall, in our previous lessons, we transformed raw text data into numerical features, for example, using the Bag-of-Words (BoW) or Term Frequency-Inverse Document Frequency (TF-IDF) techniques. These transformation methods often create what we call "Sparse Matrices," an incredibly memory-efficient way of storing high-dimensional data.

Let's break this down a bit. In the context of text data, each unique word across all documents could be treated as a distinct feature. However, each document will only include a small subset of these available features or unique words. Meaning, most entries in our feature matrix end up being 0s, hence resulting in a sparse matrix.

We'll begin with a simple non-text matrix to illustrate sparse matrices and later connect this knowledge to our journey on text data transformation.

import numpy as np
from scipy.sparse import csr_matrix, csc_matrix, coo_matrix

# Simple example matrix
vectors = np.array([
    [0, 0, 2, 3, 0],
    [4, 0, 0, 0, 6],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 7, 0, 8, 0]
])

In this section, we'll investigate how we can handle sparse matrices in different formats including: Compressed Sparse Row (CSR), Compressed Sparse Column (CSC), and the Coordinate (COO) formats.

We'll start with the CSR format, a common format for sparse matrices that is excellent for quick arithmetic operations and matrix vector calculations.

# CSR format
sparse_csr = csr_matrix(vectors)
print("Compressed Sparse Row (CSR) Matrix:\n", sparse_csr)

The output of the above code will be:

Compressed Sparse Row (CSR) Matrix:
   (0, 2)	2
  (0, 3)	3
  (1, 0)	4
  (1, 4)	6
  (4, 1)	7
  (4, 3)	8

Observe that in the output of the Compressed Sparse Row representation, it records the values in the matrix row-wise, starting from the top. Each entry (0, 2), for example, tells us that the element in the 0th row and 2nd column is 2.

Next, let's convert our vectors matrix to the CSC format. This format, like the CSR format, also forms the backbone of many operations we perform on sparse matrices. But it stores the non-zero entries column-wise, and is especially efficient for column slicing operations.

# CSC format
sparse_csc = csc_matrix(vectors)
print("Compressed Sparse Column (CSC) Matrix:\n", sparse_csc)

The output of the above code will be:

Compressed Sparse Column (CSC) Matrix:
   (1, 0)	4
  (4, 1)	7
  (0, 2)	2
  (0, 3)	3
  (4, 3)	8
  (1, 4)	6

In this Compressed Sparse Column output, the non-zero entries are stored column-wise. Essentially, CSC format is a transpose of the CSR format.