# solve an argmax problem of two unknowns given covariance matrices

The problem is: For a pair of random variable $(X,Y)$, where $X \in \mathbb{R}^p$, $Y \in \mathbb{R}^q$, write $\Sigma_{XX}=var(X)$, $\Sigma_{YY}=var(Y)$, $\Sigma_{YX}=cov(Y,X)$.

Define $(\alpha^*, \beta^*)=$ argmax $\alpha^T\Sigma_{YX}\beta$ s.t. $\alpha^T\Sigma_{YY}\alpha=1$, $\beta^T\Sigma_{XX}\beta=1$.

Assume that $\Sigma_{YY}>0$, $\Sigma_{XX}>0$. How can I solve for $(\alpha^*, \beta^*)$ if $\Sigma_{YY}$, $\Sigma_{XX}$, $\Sigma_{YX}$ are known?

• Please add the [self-study] tag (you'll have to remove an existing tag) & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. – gung Apr 4 '17 at 18:42
• – whuber Apr 4 '17 at 18:45

## 1 Answer

Recall that the maximum singular value of a matrix $A$ is the maximizer of $$\sigma_{\max}=\max_{x,y}x^{T}Ay$$

over the set $\left\Vert x\right\Vert =\left\Vert y\right\Vert =1$. The constraint sets your care about are a little different though, as they are ellipsoids rather than spheres.

The way to solve the problem you state is to apply a change of variables, changing $a\rightarrow x\Sigma_{XX}^{-1/2}$, and similarly for $b\rightarrow\Sigma_{YY}^{-1/2}y$ (we use the (principal) matrix square root here, which is well defined for positive definite covariance matrices). You then get a singular value problem of the above form which can be solved in terms of $x$ and $y$, and the solution mapped back to $a$ and $b$ vectors.