A bit of background: I am trying to create toy example of the Curds and Whey regression shrinkage algorithm in python. In a standard multivariate regression this algorithm uses canonical correlation analysis to shrink the outputs depending on their covariance with other outputs. I need to find out how to access the change of basis matrix that transforms the original matrix to the matrix in CCA space with optimal covariance.

In short, given:

import numpy as np
from sklearn.cross_decomposition import CCA

n = 100
p = 5
X = np.random.randn(n,p)
Y = np.random.randn(n,p)

cca = CCA(n_components=p)
cca.fit(X, Y)
X_c, Y_c = cca.transform(X, Y)

How do I find a (p,p) change of basis matrix 'T' such that

Y @ T == Y_c # evaluates to True, or at least are very similar
Y_c @ np.linalg.inv(T) == Y # evaluates to True, or at least are very similar

I thought about using the pseudo inverse of Y. For example if Y * T = Y_c then T = Y_si * Y_c, where Y_si is the pseudo inverse of Y. However, this gave numerically unstable results:

T = np.linalg.pinv(Y) @ Y_c

(Y_c @ np.linalg.inv(T))[:3, :3] # Should equal Y


array([[-0.13289669,  1.76632671,  0.0270406 ],
       [-1.06494794,  1.73254813,  0.21299642],
       [ 0.44353485,  0.20963675,  1.16396743]])


Y[:3, :3]


array([[-0.28513006,  1.72073957,  0.13064754],
       [-1.19718898,  1.69294781,  0.30299698],
       [ 0.30245287,  0.16738896,  1.25998495]])

Any suggestions? I am doing this in python but an example in R would be great as well.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.