How to optimize a fractional function $\frac{\left|u^HXu\right|^2}{\left|u^HYu\right|^2}$ with $\left\|u\right\|_2 = 1$ How to optimize a fractional function $\frac{|u^H X u|^2}{\left|u^HYu\right|^2}$ with $\left\|u\right\|_2 = 1$?
Here, the matrix $X \in C^{n\times n}$ is positive semidefinite, and $Y\in C^{n\times n}$ are positive definite, i.e., $X \succeq 0, Y \succ 0$.
Is there any closed form expression for $u$? 
 A: Write $f_A(u) = u^\prime A u$ for any $n\times n$ matrix $A$ and $n$-vector $u$.  It suffices to restrict attention to symmetric (real) matrices, because $f_A$ and $f_B$ with $B=(A + A^\prime)/2$ are identical forms, allowing us to replace $A$ by its symmetric version $B=B^\prime$.
Because $f_X$, $f_Y$, and $u\to ||u||^2$ are all homogeneous quadratic forms, the problem is equivalent to optimizing
$$\frac{f_X(u)/||u||^2}{f_Y(u)/||u||^2} = \frac{f_X(u)}{f_Y(u)}$$
with no constraints.  Do this by setting $f_Y(u)=1$ and maximizing $f_X(u)$, which can be accomplished by introducing a Lagrange multiplier $\lambda$.  Taking derivatives with respect to $u$ shows we need to solve the system of equations
$$uX - \lambda uY = 0.\tag{1}$$
Because $Y$ is definite, it is invertible, permitting us to right-multiply both sides of $(1)$ by $Y^{-1}$, giving
$$u(XY^{-1}) - \lambda u = 0.$$
This is the (left) eigenvalue equation for $XY^{-1}$.  Thus, the extrema are the extreme eigenvalues of $XY^{-1}$ and they are attained when $u$ is a corresponding eigenvector, scaled to unit length.

There won't generally be a closed form expression for such eigenvectors, but there is extensive software to compute them and the theory and interpretation of eigenvectors is very well established.
