Efficient Text Data Representation with Sparse Matrices | CodeSignal Learn

Efficient Text Data Representation with Sparse Matrices | CodeSignal Learn

Introduction to Sparse Matrices

Hello and welcome to this lesson on "Efficient Text Data Representation with Sparse Matrices" . As you recall, in our previous lessons, we transformed raw text data into numerical features, for example, using the Bag-of-Words (BoW) or Term Frequency-Inverse Document Frequency (TF-IDF) techniques. These transformation methods often create what we call "Sparse Matrices," an incredibly memory-efficient way of storing high-dimensional data. Let's break this down a bit. In the context of text data, each unique word across all documents could be treated as a distinct feature. However, each document will only include a small subset of these available features or unique words. Meaning, most entries in our feature matrix end up being 0s, hence resulting in a sparse matrix. We'll begin with a simple non-text matrix to illustrate sparse matrices and later connect this knowledge to our journey on text data transformation. Python import numpy as np from scipy.sparse import csr_matrix, csc_matrix, coo_matrix # Simple example matrix vectors = np.array([[0, 0, 2, 3, 0], [4, 0, 0, 0, 6], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 7, 0, 8, 0]])

Sparse Matrix Formats: CSR

In this section, we'll investigate how we can handle sparse matrices in different formats including: Compressed Sparse Row (CSR), Compressed Sparse Column (CSC), and the Coordinate (COO) formats. We'll start with the CSR format, a common format for sparse matrices that is excellent for quick arithmetic operations and matrix vector calculations. Python# CSR format sparse_csr = csr_matrix(vectors) print("Compressed Sparse Row (CSR) Matrix:\n", sparse_csr) The output of the above code will be: textCompressed Sparse Row (CSR) Matrix: (0, 2) 2 (0, 3) 3 (1, 0) 4 (1, 4) 6 (4, 1) 7 (4, 3) 8 Observe that in the output of the Compressed Sparse Row representation, it records the values in the matrix row-wise, starting from the top. Each entry (0, 2), for example, tells us that the element in the 0th row and 2nd column is 2.

Sparse Matrix Formats: CSC

Next, let's convert our vectors matrix to the CSC format. This format, like the CSR format, also forms the backbone of many operations we perform on sparse matrices. But it stores the non-zero entries column-wise, and is especially efficient for column slicing operations. Python# CSC format sparse_csc = csc_matrix(vectors) print("Compressed Sparse Column (CSC) Matrix:\n", sparse_csc) The output of the above code will be: textCompressed Sparse Column (CSC) Matrix: (1, 0) 4 (4, 1) 7 (0, 2) 2 (0, 3) 3 (4, 3) 8 (1, 4) 6 In this Compressed Sparse Column output, the non-zero entries are stored column-wise. Essentially, CSC format is a transpose of the CSR format.

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Sparse Matrix Formats: COO

Lastly, let's convert our example to the COO format or Coordinate List format. The COO format is another useful way to represent a sparse matrix and is simpler compared to CSR or CSC formats. # COO format sparse_coo = coo_matrix(vectors) print("Coordinate Format (COO) Matrix:\n", sparse_coo) The output of the above code will be: Coordinate Format (COO) Matrix: (0, 2) 2 (0, 3) 3 (1, 0) 4 (1, 4) 6 (4, 1) 7 (4, 3) 8 In the COO format, or Coordinate format, the non-zero entries are represented by their own coordinates (row, column). Unlike CSC or CSR, the COO format can contain duplicate entries. This can be particularly useful when data is being accumulated in several passes and there might be instances where duplicate entries are generated. These duplicates are not immediately merged in the COO format, providing you with flexibility for subsequent processing like duplicate resolution.

Vectorized Operations: CSR and CSC

Sparse matrices are not just memory-efficient storage mechanisms, but they also allow us to conduct operations directly on them. Specifically, the CSR and CSC formats support these operations directly, whereas the COO format requires converting back to CSR or CSC first. Let's see this in practice when performing a multiplication operation. # Running operations on CSR and CSC matrices weighted_csr = sparse_csr.multiply(0.5) print("Weighted CSR:\n", weighted_csr.toarray()) The output of the code block above will be: Weighted CSR: [[0. 0. 1. 1.5 0. ] [2. 0. 0. 0. 3. ] [0. 3.5 0. 4. 0. ] [0. 0. 0. 0. 0. ] [0. 0. 0. 0. 0. ]] We can see the impressive CSR format efficiency in vectorized operations, which becomes crucial when performing calculations with large text data.

Vectorized Operations: COO

And now let's demonstrate the process of performing the same multiplication operation on the COO format, but this time requiring conversion to CSR or CSC first. # Operation on COO requires conversion to CSR or CSC first weighted_coo = sparse_coo.tocsr().multiply(0.5) print("Weighted COO:\n", weighted_coo.toarray()) The output of the above code will be: Weighted COO: [[0. 0. 1. 1.5 0. ] [2. 0. 0. 0. 3. ] [0. 3.5 0. 4. 0. ] [0. 0. 0. 0. 0. ] [0. 0. 0. 0. 0. ]]