# How does SVD save space?

We start with an $m \times n$ matrix before SVD. After SVD, we have three matrices of sizes, $m \times m$, $n \times n$ and $m \times n$. How do we save space then if now we have three matrices instead of one and more numbers to store? Why are we talking here about dimensionality reduction? What are then the benefits of SVD?

• SVD is not about saving space but decomposition of matrix into matrices which have desirable properties like unitarity and orthonormality. It turns out you can use SVD to do dimension reduction of the data (not saving space but reducing the dimensionality) by discarding the singular values below some defined threshold. – Pankaj Daga Oct 12 '16 at 6:33
• Low-rank approximation – Rodrigo de Azevedo Oct 12 '16 at 8:20
• I think you are possibly confusing dimension here with the amount of storage space in your computer. – mdewey Oct 12 '16 at 8:47
• Suppose m x n(the total number of pixels) is fixed, and also number of singular values are fixed. What dimensions should your image have to save the most space from compression? – Simpleprint Nov 20 '16 at 14:00
• – amoeba Nov 20 '16 at 18:16

For example, here's a $512 \times 512$ B&W image of Lena:

We compute the SVD of Lena. Choosing the singular values above $1\%$ of the maximum singular value, we are left with just $53$ singular values. Reconstructing Lena with these singular values and the corresponding (left and right) singular vectors, we obtain a low-rank approximation of Lena:

Instead of storing $512^2$ values (each taking $8$ bits), we can store $2 \cdot (512 \cdot 53) + 53 = 54325$ values, which is approximately $20\%$ of the original size. This is one example of how SVD can be used to do lossy compression.

Here's the MATLAB code:

% open Lena image and convert from uint8 to double

% perform SVD on Lena
[U,S,V] = svd(Lena);

% extract singular values
singvals = diag(S);

% find out where to truncate the U, S, V matrices
indices = find(singvals >= 0.01 * singvals(1));

% reduce SVD matrices
U_red = U(:,indices);
S_red = S(indices,indices);
V_red = V(:,indices);

% construct low-rank approximation of Lena
Lena_red = U_red * S_red * V_red';

% print results to command window
r = num2str(length(indices));
m = num2str(length(singvals));
disp(['Low-rank approximation used ',r,' of ',m,' singular values']);

% save reduced Lena
imwrite(uint8(Lena_red),'Reduced Lena.bmp');

• How exactly do you get $2⋅(512⋅53)+53$ ? – sloth Nov 4 '16 at 11:48
• Okay, $(512⋅53)$ each for $U$ and $V$ and another $53$ for $S$? – sloth Nov 4 '16 at 11:50
• Yes, that is correct. – Rodrigo de Azevedo Nov 4 '16 at 11:54
• On a slightly unrelated note, how is compression possible if the image is not square, resulting in $U$ and $V$ being of different sizes? – sloth Nov 4 '16 at 14:37
• If the image is $m \times n$ and one uses $r$ singular values, then one needs to store $$m r + n r + r = (m + n + 1) r$$ values to compress the image. – Rodrigo de Azevedo Nov 4 '16 at 14:44

We talk about dimensionality reduction when you use the SVD as part of Principal Component Analysis and choose to use a subset of the available dimensions. Check out: Relationship between SVD and PCA. How to use SVD to perform PCA?

SVD can be used for an incredible amount of things, few of which relate to reducing dimensionality.

Examples of use off the top of my head: calculating pseudoinverses, solving least squares problems, analysing time-dependent concentrations in chemical reactions, X-ray scattering time series analysis, non-linear corrections on pixel array detectors, estimating the rank/finding the null space of a matrix, and inversion problems.

SVD decomposes the X matrix to U, S and V matrices where:

U * S * V' = X with V being orthagonal


There are few ways of using that matrices to reduce dimension. One of the most common ways is to choose the first desired number of columns(vectors) from The V matrix which has n x n size. Then the dimension reduction takes place as you do this:

T = X * V_with_selected_vectors


• and V matrix represents loadings The more wise tradition is to call it (right) eigenvectors. "Loadings" term is better to reserve to the up-scaled (unstandardized) eigenvectors. – ttnphns Oct 12 '16 at 7:29