16
$\begingroup$

Suppose $A$ is symmetric positive definite matrix. Is there a nice expression for the first moment of the following quantity?

$$\frac{x^TAx}{x^TA^2 x}$$

Where $x$ is distributed as $\text{Normal}(0,I_n)$. This is the ratio of two quadratic forms evaluated on the surface of the sphere.

When $A$ has eigenvalues $\langle 1, \frac{1}{2}\rangle$, this expectation is equal to $\frac{4}{3}$, visualized below (notebook)

enter image description here

Edit Aug 18 To expand on hyperplane's answer, we can take $A$ to be diagonal without loss of generality and write solution as

$$E\left[\frac{x^TAx}{x^TA^2 x}\right]=\langle a, z\rangle$$

Where $$\begin{align} a_i=&\frac{1}{A_{ii}}\\ z_i=&E_{y\sim\mathcal{N}(0,A)} \frac{y_i^2}{\|y\|^2} \end{align} $$

$\endgroup$
2
11
+50
$\begingroup$

For symmetric matrices $A, B$, the quantities $\mathcal R_A(x) = \frac{x^T A x}{x^T x}$ and $\mathcal R_{A, B}(x) = \frac{x^T Ax}{x^TBx}$ are known as the (generalized) Rayleigh quotient. A question about this was already asked here: Distribution of the Rayleigh quotient or here Expected value of Rayleigh quotient

The accepted answer refers to the 1992 book Quadratic Forms in Random Variables by Mathai and Provost.

There, on page 144, we are referred to the 1956 paper Quadratic Forms in Normally Distributed Random Variables by Gurland, where the distribution and the expectation of the generalized Rayleigh Quotient are discussed. Among other things, the author shows:

$$ \mathbf E\left[\frac{x^T Ax}{x^TBx}\right] = \sum_{j=0}^{n-1} \sum_{k=0}^{\infty} \frac{(-1)^{j+1}}{2^{j+2} v^{j+k+1}}c_{j} g_{k} B\left(j+k+1, \frac{3 n}{2}-j-1\right) $$

Here, $B(x, y)$ is the Beta Function and $v$, $c_j$ and $g_k$ are coefficients related to eigenvalues/characteristic polynomials of $A$ and $B$.

There are many other references giving different series/integral expansions/representations for the moments of $\mathcal R_{A, B}(x)$

None of which indicate that there is a general simple "closed form" for $\mathbf E[\mathcal R_{A, B}(x)]$.


Some simplification steps we can do in any case, given Eigenvalue Decomposition $B=U^T\Lambda U$:

$$ \mathbf E_{x\sim\mathcal N(0,𝕀)}\left[\frac{x^T Ax}{x^TBx}\right] = \mathbf E_{y\sim\mathcal N(0,𝕀)}\left[\frac{y^TU^T AUy}{y^T \Lambda y}\right] = \mathbf E_{z\sim\mathcal N(0,\Lambda)}\left[\frac{z^T\Lambda ^{1/2}U^T AU\Lambda ^{1/2}z}{z^T z}\right] $$

Letting $C=\Lambda ^{1/2}U^T AU\Lambda ^{1/2}$ and using linearity we have:

$$ \mathbf E_{z\sim\mathcal N(0,\Lambda)}\left[\frac{z^TCz}{z^T z}\right] = \mathbf E_{z\sim\mathcal N(0,\Lambda)}\left[\left\langle C, \;\tfrac{zz^T}{z^T z}\right\rangle\right] = \left\langle C, \; \mathbf E_{z\sim\mathcal N(0,\Lambda)}\left[\tfrac{zz^T}{z^T z}\right]\right\rangle $$

Here, both $\mathbf E_{z\sim\mathcal N(0,\Lambda)}[zz^T] = \Lambda$ and $\mathbf E_{z\sim\mathcal N(0,\Lambda)}[z^Tz] = \operatorname{tr}(\Lambda)$ are trivial, however numerical simulation suggests that $\mathbf E_{z\sim\mathcal N(0,\Lambda)}\left[\tfrac{zz^T}{z^T z}\right]$ is a diagonal matrix whose diagonal has some non-trivial, non-linear relationship w.r.t. $\Lambda$.

$\endgroup$
5
  • 1
    $\begingroup$ thanks, reading more into this. BTW, in my problem, because of rotational symmetry we can say that A is diagonal WLOG $\endgroup$ Aug 18 at 8:34
  • 1
    $\begingroup$ Using your dot product expansion simplifies things considerably, updated q $\endgroup$ Aug 18 at 9:55
  • $\begingroup$ I think that $\Lambda$ should be used instead of $\Lambda^2$ in the expectations of the numerator and denominator of $E[zz^\top/(z^\top z)]$ at the end. Maybe the partial derivatives w.r.t the $\lambda_i$ can help? $\endgroup$
    – Yves
    Aug 18 at 15:24
  • $\begingroup$ @Yves thanks for spotting that, fixed. $\endgroup$
    – Hyperplane
    Aug 19 at 8:20
  • $\begingroup$ follow-up question here -- math.stackexchange.com/questions/4228308/… $\endgroup$ Aug 19 at 16:49
2
$\begingroup$

$A\in\mathbb{R}^{n\times n}$ is symmetric positive, so there exists an orthonormal base $U=u_1,...,u_n$ and scalars $\lambda_1,...,\lambda_n$ s.t. $A=UDU^T$, with $D=\begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \\ \end{pmatrix}$. This is the spectral decomposition.

With these, we can decompose $$x^TAx=x^TUDU^Tx$$ and more importantly $$x^TA^2x=x^TUDU^TUDU^Tx$$. As U is orthonormal, $U^TU=I$ and thus we get the denominator as $x^TA^2x=x^TUD^2U^Tx$. Now we denote $w=U^Tx$. As a transformation, we get $w\sim N(U^T\mu_x, U^T\Sigma_xU)$. Plug in $x\sim N(0,I_n)$ and we get $w\sim N(0,I_n)$, again using $U^TU=I$.

The numerator is $w^TDw=\sum_{i=1}^{n}{\lambda_iw_i^Tw_i}$. Denote $g=\sum_{i=1}^{n}{\lambda_i}$, this is simply a $\chi^2_{ng}$ variable ($w^Tw\sim\chi^2_n$, we sum $g$ of those). Similarly, the denominator is a $\chi^2_{nk}$ variable, where $k=\sum_{i=1}^{n}{\lambda_i^2}$.

Overall, $\frac{x^TAx}{x^TA^2x}=\frac{w^TDw}{w^TD^2w}$ is a ratio of $\chi^2_{ng}$ variable and a $\chi^2_{nk}$ variable, so it has a beta prime distribution with parameters $\left(\alpha=\frac{ng}{2},\beta=\frac{nk}{2}\right)$. Assuming $nk>2$, the mean is

$$ \frac{\alpha}{\beta-1} = \frac{\frac{ng}{2}}{\frac{nk}{2}-1}=\frac{ng}{nk-2} $$

That's the first moment of $\frac{x^TAx}{x^TA^2x}$. Hope this helps.

$\endgroup$
7
  • 1
    $\begingroup$ I might have missed something with converting it into an $F$ variable? not sure, but the $\beta'$ and $F$ are tightly related. $\endgroup$
    – Spätzle
    Aug 12 at 22:31
  • 4
    $\begingroup$ Don't you need the two $\chi^2$ variables to be independent to get an $F$ (or a $\beta\prime$)? $\endgroup$ Aug 13 at 0:37
  • $\begingroup$ Thanks, will look at this more deeply. BTW because the problem is rotationally symmetric, could just say "A is diagonal WLOG" $\endgroup$ Aug 13 at 8:10
  • $\begingroup$ That doesn't seem to match result for A with eigenvalues 1,1/2, the result should be 4/3, verified using 3 different methods -- wolframcloud.com/obj/yaroslavvb/newton/forum-ratio-quadratic.nb $\endgroup$ Aug 13 at 8:34
  • 4
    $\begingroup$ As noted above, this fails because numerator and denominator are dependent in general. Consider for example the special case where $A$ is the identity matrix. $\endgroup$
    – ekvall
    Aug 17 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.