3
$\begingroup$

Let $X \sim \mathcal{N}\left(\mu, \Sigma \right)$, and let $A$ be a symmetric matrix. My understanding is that the Rayleigh quotient of vector $X$ is given by: $$R=\frac{X^T A X}{X^T X}$$

I've been trying unsuccessfully to find an expression for the expected value of R. This seems to be a very well studied and relatively simple random variable, so I'd expect there to be some expression for $E(R)$.

I have seen these related questions(1, 2), but these ask about the generalized Rayleigh quotient, which is more complex.

I have also looked into resources discussing the distributions of quadratic forms. From his PhD thesis beggining of section 3.2.2 (page 72), I understand that that $E(R) = \frac{E(X^T A X)}{E(X^T X)}$ (i.e. making the denominator $X^T I X$, since $I$ is idempotent as required), which seems to be off, and is in disagreement with simulations I've run. It is likely that I'm misunderstanding their claims though.

I also looked into Mathai & Provost "Quadratic forms in random variables", page 144 onwards (Ratios of Quadratic Forms), but couldn't find an answer. I may have missed the answer though, since most of the book goes over my head.

In case it's useful, this current question comes as a development of this previous question I asked here. It seems that a way to solve my previous problem is by estimating the expected value of the Rayleigh quotient.

Thanks

$\endgroup$
3
  • $\begingroup$ Can these old questions help? Question 1 and Question 2 even only one was partially answered $\endgroup$
    – jmarkov
    Feb 20, 2023 at 6:59
  • $\begingroup$ Also, you might be ok with an approximation of the ratio and someone answer a more general question about the expectation of a ratio (Question 3) $\endgroup$
    – jmarkov
    Feb 20, 2023 at 8:07
  • $\begingroup$ @jmarkov thanks. One of those questions is already linked in my question above. They ask about the generalized Rayleigh, and there's no analytic formula in the responses. I was wondering whether the Rayleigh (non-generalized) does have one. And yes, I'm already working with an approximation, but if the formula of $E(R)=\frac{E(X^T A X)}{E(X^T X)}$ above is correct (from that PhD thesis), then I could derive that my approximation is actually the analytic solution. $\endgroup$
    – dherrera
    Feb 20, 2023 at 20:11

2 Answers 2

10
$\begingroup$

I found the answer for the case where the covariance of $X$ above is the identity ($\Sigma=I$). This was enough to solve my problem of interest, so I'm posting here and voting provisionally as the correct answer, until someone comes up with the more general case (which is related to the generalized Rayleigh quotient).

I found the answer to my question in this paper. Taking Equation 2.8 in that paper and substituting q=1* gives the following expectation:

$$\mathbb{E}\left[\frac{X^T A X}{X^T X}\right] = \frac{\operatorname{tr}(A)}{n} {}_1F_1\left(1; \frac{n}{2}+1; \frac{-\Vert\mu\Vert^2}{2}\right) + \frac{1}{n+2} {}_1F_1\left(1; \frac{n}{2}+2; \frac{-\Vert\mu\Vert^2}{2}\right) \mu^T A \mu $$

for $X \sim \mathcal{N}(\mu, I)$. Here, $ {}_1F_1(a;b;c)$ is the confluent hypergeometric function, and $n$ is the number of dimensions in $X$.

I tested this expression in simulations and it seems to work perfectly. It is straightforward to extend it to a covariance that is $I\sigma^2$, where $\sigma^2$ is a scalar.

As a computational note, in case it's of use to anyone, the implementation of the confluent hypergeometric function in scipy was blowing up frequently for me. The package mpmath that allows for arbitrary precision computation has an implementation that has worked perfectly so far.

If you find this useful, consider upvoting the answer and/or question.

* The equation in the paper is more general, giving the expectation for $\mathbb{E}\left[\frac{X^T A X}{(X^T X)^q}\right]$

$\endgroup$
1
$\begingroup$

In general you will have to work with the approximation but, to my knowledge (which is not exhaustive), there is specific case for which the equality you mention can be analytically achieved. Specifically, in this paper you will find the following theorem:

Let $X:= (X_{1},..., X_{n})$ be normally distributed with $E(X)=0$ and $var(X)=\Omega$ a positive definite matrix and let $A$ be any symmetric matrix.

Under this conditions $E(\frac{X^TAX}{X^T\Omega^{-1}X})$= $\frac{E(X^TAX)}{E(X^T\Omega^{-1}X)}$ . Moreover we have that $\frac{X^TAX}{X^T\Omega^{-1}X}$ that and $X^T\Omega^{-1}X$ are independently distributed.

I could not open the thesis you link in the question, but I think this is probably the result they used.

$\endgroup$
4
  • $\begingroup$ Thanks @jmarkov . Indeed, the problem is that I need a result for E(X) different from 0, so that wouldn't work. I did find the answer to my question in an old, unknown paper. There is an easy to compute analytic expression for that expectation. When I have some time I will post the answer here. $\endgroup$
    – dherrera
    Feb 23, 2023 at 21:08
  • $\begingroup$ Yes, please, so others can also have a look at your solution! Besides, consider upvoting the answer, thank you. $\endgroup$
    – jmarkov
    Feb 24, 2023 at 7:08
  • 1
    $\begingroup$ I posted my answer, @jmarkov . I realized that I mistakenly asked the original question with non-identity covariance, while I only needed Identity covariance, so it's not actually 100% the answer to the question. However, I'm trying to take this further, and may answer the original question at some point. $\endgroup$
    – dherrera
    Feb 26, 2023 at 1:10
  • $\begingroup$ I see! That's very good though. $\endgroup$
    – jmarkov
    Feb 26, 2023 at 2:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.