residualized covariance matrix from pca/eigenvalue decomposition

I understand that given N dimensional data you can use PCA to construct an N dimensional orthonormal basis that explains 100% of the variance of the original data. However, you can also construct other orthonormal bases that explain <100% of the variance of the original data by discarding selected principal components (see How to reverse PCA and reconstruct original variables from several principal components?)

My question is assuming we have two (let's assume slightly positively correlated) sets of data $$X$$ and $$Y$$, who have sample covariance matrices $$C_x$$ and $$C_y$$. We can perform an eigenvalue decomposition on $$C_x$$ to find the orthonormal PCA basis for $$X$$. Now assuming that $$Y$$ is correlated to $$X$$, I want to find the residualized covariance matrix for $$Y$$ that removes all of the eigenbasis of $$X$$ (please bear with me - not sure if this makes sense).

So assuming that $$\lambda_x$$ is the diagonalized eigenvalue matrix of $$C_x$$ and $$V_x$$ is the stacked eigenvectors, then we know the followoing is true:

$$C_x = V_x * \lambda_x * V_x^{-1}$$

$$=> \lambda_x = V_x^{-1} * C_x * V_x$$

So I was thinking that we could find some analogue to the eigenvalue loadings of $$C_y$$ on the basis of $$X$$ by doing:

$$\lambda_y^* = V_x^{-1} * C_y * V_x$$

I know that $$\lambda_y^*$$ likely will not have the structure of a covariance matrix, but does this quantity have any meaning? Is there a way to use this quantity to determine how much of $$C_y$$ is explained by the eigenbasis of $$C_x$$?

• Perhaps PLS regression would be of more interest to you. – Carl Nov 26 '18 at 21:06