Singular Value Decomposition, commonly abbreviated as SVD, is a matrix factorization technique in linear algebra. SVD decomposes a matrix into three other matrices, capturing important properties of the original matrix.
For instance, if you have a matrix A, the SVD is given by −
A = UΣVT
Here, U and V are orthogonal matrices, and Σ is a diagonal matrix.
The columns of U are called the left singular vectors, the columns of V (or rows of VT) are the right singular vectors, and the entries of Σ are the singular values.
NumPy provides the numpy.linalg.svd() function to compute the Singular Value Decomposition of a matrix. Let us see how to use this function with an example.
In this example, the matrix A is decomposed into three matrices: U, Σ (represented as the array of singular values S), and VT −
# Open Compiler
import numpy as np
# Define a 3x3 matrix
A = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(A)
print("Matrix U:\n", U)
print("Singular values:", S)
print("Matrix V^T:\n", VT)
Output:
Following is the output obtained −
Matrix U:
[[-0.21483724 0.88723069 0.40824829]
[-0.52058739 0.24964395 -0.81649658]
[-0.82633754 -0.38794278 0.40824829]]
Singular values: [1.68481034e+01 1.06836951e+00 4.41842475e-16]
Matrix VT:
[[-0.47967118 -0.57236779 -0.66506441]
[-0.77669099 -0.07568647 0.62531805]
[-0.40824829 0.81649658 -0.40824829]]
The SVD components have specific properties and roles as shown below −
You can reconstruct the original matrix A from its SVD components. In NumPy, you can achieve this by using the numpy.dot() function to perform matrix multiplication.
In the following example, we are reconstructing the original matrix “A” −
# Open Compiler
import numpy as np
# Define a 3x3 matrix
A = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(A)
# Create the diagonal matrix Σ from the singular values
Sigma = np.zeros((3, 3))
np.fill_diagonal(Sigma, S)
# Reconstruct the original matrix
A_reconstructed = np.dot(U, np.dot(Sigma, VT))
print("Original matrix:\n", A)
print("Reconstructed matrix:\n", A_reconstructed)
Output:
The original matrix A is successfully reconstructed using its SVD components, demonstrating the accuracy of the decomposition −
Original matrix:
[[1 2 3]
[4 5 6]
[7 8 9]]
Reconstructed matrix:
[[1. 2. 3.]
[4. 5. 6.]
[7. 8. 9.]]
SVD is a powerful tool with numerous applications, such as −
Let us see an example of how SVD can be used for image compression. We will use a grayscale image and compress it by retaining only the most significant singular values −
import numpy as np
import matplotlib.pyplot as plt
from skimage import data, color
# Load a sample image and convert it to grayscale
image = color.rgb2gray(data.astronaut())
# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(image, full_matrices=False)
# Retain only the first k singular values
k = 50
U_k = U[:, :k]
S_k = np.diag(S[:k])
VT_k = VT[:k, :]
# Reconstruct the compressed image
image_compressed = np.dot(U_k, np.dot(S_k, VT_k))
# Plot the original and compressed images
plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.title("Original Image")
plt.imshow(image, cmap='gray')
plt.axis('off')
plt.subplot(1, 2, 2)
plt.title(f"Compressed Image with k={k}")
plt.imshow(image_compressed, cmap='gray')
plt.axis('off')
plt.show()
Output:
The original image and the compressed image are displayed side by side, demonstrating how SVD can reduce the size of the image while preserving its essential features −
SVD provides several advantages, such as −
Key Takeaway: Master Singular Value Decomposition with NumPy at Vista Academy!