# Keeping starting variables after a Principal component analysis

I know the PCA theory and how keeping the k-first principal components help dimensionality reduction. But what about keeping the base vectors ?

You start with $$n$$ row vectors $$x_1,...,x_n$$, standardize them, then find the covariance matrix $$C$$ and diagonalise it $$C=VDV^{-1}$$, with $$D_{ij}=\lambda_j\delta_{ij}$$. Then after ordering the eigenvector with decreasing eigenvalue order you can use the first $$p$$ eigenvectors as a base to express your starting variables. The process will ensure you explain a maximum of the variance while only keeping $$p variables. The main downside I see to the process is the somewhat lack of transparency of the eigenvectors, starting variables have values, units, meanings, that are usefull to deal with on a daily basis.

Is there any way to express the proportion of the variance explained by one of the starting vector ? by a subset of $$p$$ of the starting vectors ?

Is there any process to select $$p$$ of the starting vectors to maximise the variance explained by this subset ? Would this kind of process be incremental ? (ie. can the selection of $$p+1$$ vectors that maximise the explained variance can be built by adding one vector to the selection of $$p$$ vectors that maximised the explained variance ?).

• You are very nearly describing stepwise regression, so that's a good term to research for answers. – whuber Feb 13 at 22:38
• Ok thanks, i'll look into that. What about my first question ? Is it possible to express the proportion of variance explained by a subset of variables ? – were_cat Feb 14 at 10:05
• That's exactly what $R^2$ does. – whuber Feb 14 at 15:36