0
$\begingroup$

A bit of background: I am trying to create toy example of the Curds and Whey regression shrinkage algorithm in python. In a standard multivariate regression this algorithm uses canonical correlation analysis to shrink the outputs depending on their covariance with other outputs. I need to find out how to access the change of basis matrix that transforms the original matrix to the matrix in CCA space with optimal covariance.

In short, given:

import numpy as np
from sklearn.cross_decomposition import CCA


n = 100
p = 5
X = np.random.randn(n,p)
Y = np.random.randn(n,p)

cca = CCA(n_components=p)
cca.fit(X, Y)
X_c, Y_c = cca.transform(X, Y)

How do I find a (p,p) change of basis matrix 'T' such that

Y @ T == Y_c # evaluates to True, or at least are very similar
Y_c @ np.linalg.inv(T) == Y # evaluates to True, or at least are very similar

I thought about using the pseudo inverse of Y. For example if Y * T = Y_c then T = Y_si * Y_c, where Y_si is the pseudo inverse of Y. However, this gave numerically unstable results:

T = np.linalg.pinv(Y) @ Y_c

(Y_c @ np.linalg.inv(T))[:3, :3] # Should equal Y

Outputs:

array([[-0.13289669,  1.76632671,  0.0270406 ],
       [-1.06494794,  1.73254813,  0.21299642],
       [ 0.44353485,  0.20963675,  1.16396743]])

while

Y[:3, :3]

Outputs:

array([[-0.28513006,  1.72073957,  0.13064754],
       [-1.19718898,  1.69294781,  0.30299698],
       [ 0.30245287,  0.16738896,  1.25998495]])

Any suggestions? I am doing this in python but an example in R would be great as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.