The problem is: For a pair of random variable $(X,Y)$, where $X \in \mathbb{R}^p$, $Y \in \mathbb{R}^q$, write $\Sigma_{XX}=var(X)$, $\Sigma_{YY}=var(Y)$, $\Sigma_{YX}=cov(Y,X)$.

Define $(\alpha^*, \beta^*)=$ argmax $\alpha^T\Sigma_{YX}\beta$ s.t. $\alpha^T\Sigma_{YY}\alpha=1$, $\beta^T\Sigma_{XX}\beta=1$.

Assume that $\Sigma_{YY}>0$, $\Sigma_{XX}>0$. How can I solve for $(\alpha^*, \beta^*)$ if $\Sigma_{YY}$, $\Sigma_{XX}$, $\Sigma_{YX}$ are known?

  • 1
    $\begingroup$ Please add the [self-study] tag (you'll have to remove an existing tag) & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$ – gung - Reinstate Monica Apr 4 '17 at 18:42
  • 1
    $\begingroup$ See en.wikipedia.org/wiki/…. $\endgroup$ – whuber Apr 4 '17 at 18:45

Recall that the maximum singular value of a matrix $A$ is the maximizer of $$ \sigma_{\max}=\max_{x,y}x^{T}Ay $$

over the set $\left\Vert x\right\Vert =\left\Vert y\right\Vert =1$. The constraint sets your care about are a little different though, as they are ellipsoids rather than spheres.

The way to solve the problem you state is to apply a change of variables, changing $a\rightarrow x\Sigma_{XX}^{-1/2}$, and similarly for $b\rightarrow\Sigma_{YY}^{-1/2}y$ (we use the (principal) matrix square root here, which is well defined for positive definite covariance matrices). You then get a singular value problem of the above form which can be solved in terms of $x$ and $y$, and the solution mapped back to $a$ and $b$ vectors.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.