Singular Value Decomposition (SVD) with NumPy

Welcome to the lesson! In this lesson, we will explore Singular Value Decomposition (SVD), a powerful tool in linear algebra with significant applications in data science and machine learning. Although we won't delve into the detailed mathematics behind SVD, it's important to know that SVD is a method of decomposing a matrix into three other matrices, providing new perspectives on the data. Real-world applications of SVD include tasks such as image compression, noise reduction, and dimensionality reduction.

Singular Value Decomposition decomposes a given matrix ( A ) into three simpler matrices: ( U ), ( S ), and ( V^T ). Here's a brief overview of these components:

• U Matrix: An orthogonal matrix that represents the left singular vectors.
• Singular Values (S): These are diagonal elements and represent the magnitudes of the axes.
• V Transpose Matrix (V^T): Another orthogonal matrix representing the right singular vectors.

Imagine each matrix as playing a role in a transformation. By breaking down ( A ) into these components, we can analyze and manipulate the matrix in more intuitive ways.

Let's walk through an example to see how Singular Value Decomposition is performed using NumPy. We'll use the following matrix and decompose it using NumPy's SVD function:

import numpy as np

# Defining a matrix
matrix = np.array([[1, 2, 3], [4, 5, 6]])

# Singular Value Decomposition
U, S, Vt = np.linalg.svd(matrix)

# Display results
print("Matrix:\n", matrix)
print("U Matrix:\n", U)
print("Singular Values:", S)
print("V Transpose Matrix:\n", Vt)

# Output:
# Matrix:
#  [[1 2 3]
#  [4 5 6]]
# U Matrix:
#  [[-0.3863177  -0.92236578]
#  [-0.92236578  0.3863177 ]]
# Singular Values: [9.508032   0.77286964]
# V Transpose Matrix:
#  [[-0.42866713 -0.56630692 -0.7039467 ]
#  [ 0.80596391  0.11238241 -0.58119967]
#  [ 0.40824829 -0.81649658  0.40824829]]

1. Defining a Matrix: We start by defining a simple 2x3 matrix using np.array. This matrix will be the one we decompose using SVD.

2. Performing SVD: We call np.linalg.svd(matrix), a function in NumPy that computes the Singular Value Decomposition of the matrix. It returns U, S, and Vt.

3. Output: The matrices U and Vt, along with the singular values contained in S, are printed:

   • The U Matrix contains the left singular vectors.
   • The Singular Values (S) are shown as a 1D array; these values signify the importance or strength of each corresponding dimension.
   • The V Transpose (Vt) Matrix contains the right singular vectors.