PCA on data having more dimensions than observations [duplicate]

I have a 73 by 426 matrix, where 73 is the number of my observations, and 426 is the number of my features/dimensions. When I perform the PCA, I expect to get a 73 by 426 score matrix; instead, I get a 73 by 72 matrix. I assume this is because I have more dimensions than observations. Is there a way to overcome this problem?

Here is my code in Matlab:

[coeff,score,~,~,explained] = pca(class1Sgn);

marked as duplicate by gung♦, kjetil b halvorsen, usεr11852, SmallChess, JohnKApr 7 '17 at 7:54

• Partial least squares is a kind of predictive PCA useful in your specific instance with more features than data. – Mike Hunter Apr 6 '17 at 16:29
• Is this a pca function you wrote? Some environment details would be helpful. – Matt L. Apr 6 '17 at 16:31
• no this is the function provided by MATLAB – xava Apr 6 '17 at 16:47
• why do you expect 73 by 426 score matrix? – Aksakal Apr 6 '17 at 17:15
• I think you will find the information you need in the linked thread. Please read it. If it isn't what you want / you still have a question afterwards, come back here & edit your question to state what you learned & what you still need to know. Then we can provide the information you need without just duplicating material elsewhere that already didn't help you. – gung Apr 6 '17 at 20:08

Took some digging, but if you look at the MATLAB documentation it uses the SVD function to accomplish this task. If you look at the SVD documentation it states:

https://www.mathworks.com/help/matlab/ref/svd.html

[m-by-n matrix A]

m > n — Only the first n columns of U are computed, and S is n-by-n.

m = n — svd(A,'econ') is equivalent to svd(A).

m < n — Only the first m columns of V are computed, and S is m-by-m.

And then if you take what @user3348782 said you lose 1 degree of freedom when you do PCA and the result is a m-by-(m-1) matrix.

Recall that principal components are, by construction, orthogonal. Your original data has a rank of 73 at most, so you cannot derive more than 73 principal components from it. In fact you will lose a degree of freedom yielding 72 PC's.

But what in the world do you plan on doing with 72 principal components?

I can't suggest whether to go this route without knowing your use case, but using a handful of principal components (the first 5-10 for instance) out of your 72 is done in some cases. Things can go wrong in PCA, even for the first few PC's, if your eigenvalues/scree plot do not show those first PC's having much larger eigenvalues than the others.

There is no way for PCA to give you a 73 by 436 score matrix. You could force factor analysis to do it, but I don't think you would yield anything useful. If going the PCA route, you could bootstrap several estimates to test the stability of your first few principal components.

https://arxiv.org/pdf/0911.3827.pdf

https://stats.stackexchange.com/a/45859/69090