Linked Questions

340
votes
3answers
212k views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
65
votes
18answers
87k views

Statistics interview questions

I am looking for some statistics (and probability, I guess) interview questions, from the most basic through the more advanced. Answers are not necessary (although links to specific questions on this ...
9
votes
3answers
7k views

What is the point of singular value decomposition?

I don't understand why reduction in dimension is important. What is the benefit of taking some data and reducing their dimension?
3
votes
3answers
2k views

How does SVD save space?

We start with an $m \times n$ matrix before SVD. After SVD, we have three matrices of sizes, $m \times m$, $n \times n$ and $m \times n$. How do we save space then if now we have three matrices ...
2
votes
1answer
1k views

What is the PCA representation of an image?

I've read a lot about how PCA is used to reduce the dimensionality of data, including this great answer and this mathy post. But I'm unclear what this does when applied to image data? For example, ...
2
votes
1answer
535 views

Confirming an understanding of SVD

I'm trying to sift through the concepts of low-rank approximations, matrix factorizations and SVD. There's a lot of info out there and the rabbit hole is deep so I just want to make sure my high level ...
3
votes
0answers
1k views

Principal Component Analysis: how to interpret the total contribution of variables on several dimensions

When we calculate the total contribution of a variable for a single dimension, the sum of all single contributions is equal to 100%, which makes perfect sense. The http://www.sthda.com suggests to ...
2
votes
3answers
162 views

Importance of complicated linear algebra for data scientist [closed]

I've already finished Andrew Ng's machine learning course and now working with textbook 'The Elements of Statistical Learning'. I'm successfully implementing equations and concepts described there ...
2
votes
1answer
202 views

When can I use $XX^\top/n$ as covariance matrix for PCA?

Given a data matrix $\mathbf X$, can I always obtain its covariance matrix (to use in PCA) by centering (subtracting the column means) and then computing $\mathbf X \mathbf X^\top/n$? Is this always ...
2
votes
0answers
176 views

Why PMI + SVD works for similarities arithmetics?

Recently Julia Silge blogged here and here, quoting blog entry by Chris Moody, who suggested that the similarities arithmetic in word2vec can be approximated by using PMI indexes followed by SVD ...
2
votes
0answers
58 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...