# Variant of principal components analysis where only the variance of a single variable counts

Principal components analysis finds the axis(/es) along which the variance of some dataset is maximized. What if I'm in a situation where I have a single dependent variable $$v$$ that is dependent on $$n$$ independent variables $$v = f(x_1, x_2, \dots, x_n)$$ and I would like to know "what principal components of $$\mathbf x$$ cause the most variance in $$v$$?" Is this an answered question already? I feel like it would be just a small modification of the PCA algorithm..

• The answer is that none of the PCs of $X$ may be associated with $v.$ Ordinary least squares regression can be interpreted as finding a linear combination of columns of $x$ that maximize the correlation with $v.$
– whuber
Sep 1, 2019 at 21:43
• @whuber is it exactly the same problem as iterated OLS then? Sep 1, 2019 at 21:56
• @whuber also, to clarify a possible misunderstanding - by "principal components of $\mathbf x$" I do not mean the p.c.s as identified by PCA, rather I mean "set of axes that give directions that drive the most variance of $v$". This is probably related to the output of ordinary least squares regression. Sep 1, 2019 at 22:05

If I got your question correctly (and I am not sure), the solution is that you have to take the spectral decomposition $$VDV^{T}$$ where V is the orthonormal matrix of eigenvectors and D is the diagonal matrix of eigenvalues. Then you transform the set of dependent variables by multiplying the original matrix of predictors $$X^{*}$$ by the eigenvectors $$X=X^{*}V$$. Then you can regress y against the new transformer predictors X that are now uncorrelated. Since they are uncorrelated, the total explained variance of y will be the sum of variances of $$\beta x$$ where each x is the vector representing each column of X (so each transformed variable) and $$\beta$$ denotes its corresponding beta estimated at previous step. The PC variable $$x^{max}$$ such that $$\beta^{2}_{x^{max}} Var(x^{max})$$ is maximum among all the pc transformed variables is the one explaining the highest portion of the variance of $$y$$.