I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points are stacked into a matrix $X$ with $n$ rows and $p$ columns.
What is the fastest way to do it? In fact, I am only interested in the signs of PC1 values (i.e. which of the $n$ points have positive sign and which ones have negative sign).
We could multiply $X^T X$ to get the $p \times p$ covariance matrix $C$, then find all its eigenvectors, discard all except the top one $w$, and use that as the PC1 axis to project our points onto, to get $t_1$, then take the sign of the result. Finding only the first axis then trivially supports finding the first component score of each of the points.
Alternatively, we could take the SVD of $X$ to form matrices $U, W,$ and $V^T$, with $X = U W V^T$; where $U$ is $n\times p$ and is the new coordinates; $W$ is a $p\times p$ diagonal matrix of the eigenvalues; $V^T$ is a $p\times p$ matrix of all the eigenvectors; we then throw away everything except the first column in $U$, giving a column of first components $t_1$, and use the sign of each member for the results.
Both of these process huge data which then gets discarded, and seem at least $O(n*p^2)$. What's the preferred, fastest, robust, reliable way to find just the first Principal Component's scores $t_1$? Deterministic preferred but approximations allowed. (Math, pseudocode, Numerical Recipes C++, or C++; no R nor MATLAB kindly please). Thanks!
Ed.: To clarify the problem, start with e.g. a cloud of 100 points in 4 dimensions $<i,j,k,l>$. Find the PCA of the system, giving a new coordinate system basis $<w_1, w_2, w_3, w_4>$ composed of the orthogonal unit eigenvectors of the cloud, ordered by importance. Each of the 100 points will then have a new coordinate $<t_1, t_2, t_3, t_4>$ being that point's projection (score) or "component" along the first Principal Component axis, the second Principal Component axis, etc.
But this is a huge amount of information to derive, as we only want the $t_1$ coordinate (score) for each point. So now throw away all the rest, giving 100 points of $<$t_1$, ?, ?, ?>$. Alternatively, we could merely find the first eigenvector $w_1$, skipping the machinery for $w$'s 2 thru p, and then simply project all points onto this single axis.
Now go one step further, and simply find $<sgn(t_1)>$ of this coordinate for each of 100 points.
There is apparently an issue that the sign of a $w$ eigenvector is essentially arbitrary, as its negation also forms a perfectly good basis vector. In this case, fix the $w$ vector so that it points either one way or the other, and run all the points against this axis. The sign is arbitrary, but the results for the point cloud will be mutually consistent. As the $C$ matrix is positive, and the PCA math comes out with repeatable results, I suppose this should be a non-issue in most practical cases.