Determinant and Linear Dependency

Determinant and Linear Dependency | CodeSignal Learn

Lesson Introduction

Welcome to our lesson on Determinants and Linear Dependency! These concepts are very important in machine learning. In this lesson, we will cover: What a determinant is and how to calculate it. What linear dependency means. How to check for linear dependency using determinants. By the end of this lesson, you’ll be ready to apply these concepts. Let’s dive in!

Determinants in Linear Algebra

2 \times 2

Determinant Calculation in Python

2 \times 2

Concept of Linear Dependency

v_1 = [1, 2]

Checking Linear Dependency

v_1

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

A determinant is a special number calculated from a square matrix (equal number of rows and columns). The determinant helps us understand properties, such as whether a matrix is invertible.

For a $2 \times 2$ matrix $A$ :

$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$

The determinant, denoted as det(A), is calculated as:

$det(A) = a \cdot d - b \cdot c$

For larger matrices, the determinant is calculated using a recursive process involving minors and cofactors, which generalizes the concept of the $2 \times 2$ determinant. We won't go this deep into details in this course, as the calculation itself is not important for machine learning – the key is to understand the idea of the determinant.

Linear dependency means one vector in a set can be written as a combination of the others. For example:

$v_1 = [1, 2]$ $v_2 = [2, 4]$

Here, $v_2$ is just twice $v_1$ , showing the vectors are linearly dependent.

If any matrix row is linearly dependent of any other matrix row, or if any matrix column is linearly dependent of any other matrix column, such matrix will always have a determinant of 0.

Let's check it by constructing a matrix from linearly dependent $v_1$ and $v_2$ :

Python

import numpy as np

matrix = np.array([
    [1, 2],
    [2, 4],
])

print(np.linalg.det(matrix))  # 0.0

This matrix has linear dependent rows, so its determinant is zero.

One practical application of determinants is in solving systems of linear equations. Determinants can be used to find the inverse of a matrix, which is then applied in various algorithms, including those used in computer graphics transformations and machine learning optimizations.

Consider the following system of linear equations:

\begin{align*} x + 2y &= 4 \\ 2x + 4y &= 10 \end{align*}

And here is its matrix form:

\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 4 \\ 10 \end{bmatrix}

If you try to solve this equation, you will see that it is not possible. It happens because the equations are linearly dependent. For large equation systems it can be not so obvious, and calculating the matrix' determinant might be helpful in determining if the system has a solution.