5
$\begingroup$

I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points are stacked into a matrix $X$ with $n$ rows and $p$ columns.

What is the fastest way to do it? In fact, I am only interested in the signs of PC1 values (i.e. which of the $n$ points have positive sign and which ones have negative sign).


We could multiply $X^T X$ to get the $p \times p$ covariance matrix $C$, then find all its eigenvectors, discard all except the top one $w$, and use that as the PC1 axis to project our points onto, to get $t_1$, then take the sign of the result. Finding only the first axis then trivially supports finding the first component score of each of the points.

Alternatively, we could take the SVD of $X$ to form matrices $U, W,$ and $V^T$, with $X = U W V^T$; where $U$ is $n\times p$ and is the new coordinates; $W$ is a $p\times p$ diagonal matrix of the eigenvalues; $V^T$ is a $p\times p$ matrix of all the eigenvectors; we then throw away everything except the first column in $U$, giving a column of first components $t_1$, and use the sign of each member for the results.

Both of these process huge data which then gets discarded, and seem at least $O(n*p^2)$. What's the preferred, fastest, robust, reliable way to find just the first Principal Component's scores $t_1$? Deterministic preferred but approximations allowed. (Math, pseudocode, Numerical Recipes C++, or C++; no R nor MATLAB kindly please). Thanks!


Ed.: To clarify the problem, start with e.g. a cloud of 100 points in 4 dimensions $<i,j,k,l>$. Find the PCA of the system, giving a new coordinate system basis $<w_1, w_2, w_3, w_4>$ composed of the orthogonal unit eigenvectors of the cloud, ordered by importance. Each of the 100 points will then have a new coordinate $<t_1, t_2, t_3, t_4>$ being that point's projection (score) or "component" along the first Principal Component axis, the second Principal Component axis, etc.

But this is a huge amount of information to derive, as we only want the $t_1$ coordinate (score) for each point. So now throw away all the rest, giving 100 points of $<$t_1$, ?, ?, ?>$. Alternatively, we could merely find the first eigenvector $w_1$, skipping the machinery for $w$'s 2 thru p, and then simply project all points onto this single axis.

Now go one step further, and simply find $<sgn(t_1)>$ of this coordinate for each of 100 points.

There is apparently an issue that the sign of a $w$ eigenvector is essentially arbitrary, as its negation also forms a perfectly good basis vector. In this case, fix the $w$ vector so that it points either one way or the other, and run all the points against this axis. The sign is arbitrary, but the results for the point cloud will be mutually consistent. As the $C$ matrix is positive, and the PCA math comes out with repeatable results, I suppose this should be a non-issue in most practical cases.

$\endgroup$
  • 4
    $\begingroup$ You can use the power method to get the first principal component only. $\endgroup$ – seanv507 Feb 21 '17 at 6:21
  • 2
    $\begingroup$ The power method applied to the covariance matrix can find the first principal component. But the sign of a principal component is arbitrary, so the question has no meaningful answer. $\endgroup$ – Gordon Smyth Feb 21 '17 at 6:24
  • 1
    $\begingroup$ Could you please specify what do you mean by sign of principal Component. The co-variance matrix is positive semi definite and hence all of its eigen values are non negative. As specified you can use the Power iteration method to compute the the most dominant eigen vector(the one having Eigen value of maximum absolute norm) of the co-variance matrix. This would give you the 1st principal component. $\endgroup$ – Santanu_Pattanayak Feb 21 '17 at 6:33
  • 2
    $\begingroup$ The question is unclear. Find the PC1 or its "sign"? And what is its sign? $\endgroup$ – ttnphns Feb 21 '17 at 7:41
  • 1
    $\begingroup$ After the edit, the question is pretty clear, I vote to reopen. $\endgroup$ – amoeba Feb 21 '17 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.