2
$\begingroup$

I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low pass filter.

So, before we discuss SVD, I want to check if my understanding of eigenvalue decomposition is correct.

So, the eigenvalue decomposition of a square matrix can be written as:

$$A = VDV^{-1}$$

Where $V$ is the matrix where each column corresponds to an eigenvector of $A$ and $D$ is the diagonal matrix where the diagonal entry corresponds to the corresponding eigenvector. So, I can perform compression using eigenvalue decomposition by setting the eigenvalues under some threshold to 0. In this case, the smaller eigenvalues will have a relatively shrinking effect on the rows of $V^{-1}$ and will overall contribute less. I am hoping this reasoning is correct?

The main question that I have and this relates to eigenvalue decomposition as well as SVD is whether there is some relationship to the frequency content of the signal. Do the smaller eigenvalues contribute to high-frequency components of the signal? So is the compression algorithm acting like a low pass filter and depending on the threshold set is essentially stopping the high frequency signals to pass through and basically acting as a smoothing oiperator or is there no relationship there?

$\endgroup$
1
$\begingroup$

Image filtering (in the frequency domain) is performed by decomposing an image in its frequency components and removing part of the spectrum. In this sense both SVD and image filtering perform a decomposition on images based on a change of basis.

The similarity between both techniques stops there as they operate over different domains. SVD performs a decomposition based on the spatial structure of a matrix (image) whereas a spectral filters look at its frequency components.

To illustrate the point let's see how both approaches are applied to a very simple image: enter image description here

I'll use the $A = USV^\intercal$ notation for SVD (notice that $U$ is actually a different matrix to $V^\intercal$ as in your notation).

$U$ operates in the column space of $A$ and $V$ in its row space. For the previous image we have a perfect decomposition for k-rank equals to 1. Which means that we only need to keep the first row of $U$ to recover the image ($V$ doesn't contain any information, as the image has no variability in the horizontal axis).

If we now look at the transpose of the first image $A^\intercal$, $U$ and $V$ change places in the equation following the rules of transposition of products and we'll have the opposite case: we only need to keep the first column of $V$ to recover the image ($U$ doesn't contain any information, as the image has no variability in the vertical axis).

enter image description here

The "equivalent" operation to lower the rank in SVD decomposition would be to apply a low pass filter in the frequency domain. The result for the previous image would be a low frequency version which looks like a "blurry" version of the initial image:

enter image description here

To sum up, the relationship between SVD and spectral decomposition of images is vague. SVD provides a linear decomposition of the vertical and horizontal features of an image, in which each of the components (ranks) can contain high and low frequencies. The more vertical and horizontal variability in the image the more components you'll need in order to reproduce the image.

In terms of data compression, both domains offer competitive approaches depending on the case, there is no one better than the other. This is currently a quite exciting field of research applied to multidimensional arrays (tensors).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.