A bit of background: I am trying to create toy example of the Curds and Whey regression shrinkage algorithm in python. In a standard multivariate regression this algorithm uses canonical correlation analysis to shrink the outputs depending on their covariance with other outputs. I need to find out how to access the change of basis matrix that transforms the original matrix to the matrix in CCA space with optimal covariance.
In short, given:
import numpy as np from sklearn.cross_decomposition import CCA n = 100 p = 5 X = np.random.randn(n,p) Y = np.random.randn(n,p) cca = CCA(n_components=p) cca.fit(X, Y) X_c, Y_c = cca.transform(X, Y)
How do I find a (p,p) change of basis matrix 'T' such that
Y @ T == Y_c # evaluates to True, or at least are very similar Y_c @ np.linalg.inv(T) == Y # evaluates to True, or at least are very similar
I thought about using the pseudo inverse of Y. For example if Y * T = Y_c then T = Y_si * Y_c, where Y_si is the pseudo inverse of Y. However, this gave numerically unstable results:
T = np.linalg.pinv(Y) @ Y_c (Y_c @ np.linalg.inv(T))[:3, :3] # Should equal Y
array([[-0.13289669, 1.76632671, 0.0270406 ], [-1.06494794, 1.73254813, 0.21299642], [ 0.44353485, 0.20963675, 1.16396743]])
array([[-0.28513006, 1.72073957, 0.13064754], [-1.19718898, 1.69294781, 0.30299698], [ 0.30245287, 0.16738896, 1.25998495]])
Any suggestions? I am doing this in python but an example in R would be great as well.