7
$\begingroup$

I have a multidimensional random variable $X \sim \mathcal{N} \left(\mu, I_d \right)$.

Ideally, I would like to know the expected value of the normalized outer product of the latent variable with itself: $ \mathbb{E}\left( \frac{X}{\lVert X \rVert_2} \cdot \frac{X^T}{\lVert X \rVert_2} \right) $

Alternatively, as an approximation, I'd be happy with using: $X \cdot X^T \mathbb{E} \left(\frac{1}{\lVert X \rVert_2^2} \right)$

However, I can't find an answer of what is $\mathbb{E}\left( \frac{1}{\lVert X \rVert_2^2 }\right)$ (i.e. expected value of inverse squared L2 norm) for the non-centered Gaussian case. This question is related (as well as other questions), but it asks for only 0 mean random variables. I can't seem to find an answer for the non-centered case.

Any further approximation that seems sensible would be appreciated.

Thanks!

$\endgroup$
7
  • 2
    $\begingroup$ Is it a acceptable for you to numerically integrate with respect to the density of the noncentral chi-squared distribution? $\endgroup$ Commented Feb 16, 2023 at 6:53
  • 1
    $\begingroup$ It helps to simplify your question first. Without any loss of generality you may choose coordinates in which $\mu=(m,0,\ldots,0)$ and write $X=\mu+Z$ where $Z$ is standard multivariate Normal. This leads to four kinds of expectations: $E((m+Z_1)^2/((m+Z_1)^2+Z_2^2+\cdots+Z_d^2)),$ $E(Z_2^2/((m+Z_1)^2+Z_2^2+\cdots+Z_d^2)),$ $E((m+Z_1)Z_2/((m+Z_1)^2+Z_2^2+\cdots+Z_d^2)),$ and $E(Z_2Z_3/((m+Z_1)^2+Z_2^2+\cdots+Z_d^2))$ The symmetry of the marginals of $Z$ implies the latter two are zero. The first two are closely related to non-central F distributions. $\endgroup$
    – whuber
    Commented Feb 16, 2023 at 16:51
  • $\begingroup$ @YashaswiMohanty, that would not be acceptable, but I hadn't thought of using the noncentral chi-squared distribution (I'm not a statistician). You're comment had me finding this post which may solve my simplified problem. stats.stackexchange.com/questions/374315/… $\endgroup$
    – dherrera
    Commented Feb 16, 2023 at 18:49
  • $\begingroup$ Thanks @whuber , that sounds very useful, and on point to what I want. I'll see if I can turn that into a solution of the problem and then answer the question myself. Although be my guest if you want to post your comment (or an ellaboration) as an answer, which I'll mark as the solution for now. $\endgroup$
    – dherrera
    Commented Feb 16, 2023 at 18:50
  • $\begingroup$ @whuber to ellaborate on your answer, I see two differences between the first two types of expectations and the non-central F distribution: 1) the numerator and denominator are not independent in this case, being that they share some 𝑍𝑖, 2) The denominator is non-central in this case too, unlike the non-central F distribution (en.wikipedia.org/wiki/Noncentral_F-distribution). Does this sound right? $\endgroup$
    – dherrera
    Commented Feb 16, 2023 at 19:42

1 Answer 1

9
$\begingroup$

After some work I found the solution to the problem. The solution is the following:

Eq (1). \begin{equation} \mathbb{E}\left[\frac{XX^T}{X^TX}\right] = \frac{1}{n}{}_{1}F_1\left(1; \frac{n}{2}+1; \frac{-||\mu||^2}{2}\right)I + \frac{1}{n+2} {}_{1}F_1\left(1; \frac{n}{2}+2; \frac{-||\mu||^2}{2}\right)\mu\mu^T \end{equation}

for $X\sim\mathcal{N}(\mu, I)$, where $I$ in the formula is the identity matrix, $n$ is the dimensionality of $X$, and ${}_{1}F_1(a;b;c)$ is the confluent hypergeometric function. So, the solution is a weighted sum of the identity matrix and the outer product of the variable mean, with the weights given by the hypergeometric confluent function and dimension of $X$.

The derivation is as follows:

A well studied, related problem, is to find the expectation of ratios of quadratic forms $\mathbb{E}\left[\frac{X^TAX}{X^TX}\right]$, where $X\sim\mathcal{N}(\mu,I)$ and $A$ is a symmetric matrix. In this post, I had asked for an analytic formula to find this last expectation, and I then posted an answer I found. The formula is:

Eq (2). \begin{equation} \mathbb{E}\left[\frac{X^T A X}{X^T X}\right] = \frac{tr(A)}{n} {}_1F_1\left(1; \frac{n}{2}+1; \frac{-||\mu||^2}{2}\right) + \frac{1}{n+2} {}_1F_1\left(1; \frac{n}{2}+2; \frac{-||\mu||^2}{2}\right) \mu^T A \mu \end{equation}

Now, we can use this formula to estimate each individual element $i,j$ of the original expectation by finding a matrix $A$ such that: $$\mathbb{E}\left[\frac{XX^T}{X^TX}\right]_{i,j} = \mathbb{E}\left[\frac{X_iX_j}{X^TX}\right] = \mathbb{E}\left[\frac{X^T A X}{X^T X}\right]$$

It is easy to see that this relation is satisfied for each element $i,j$ by the following matrices:

  • When $i=j$, $A_{i,i}=1$ and 0 elsewhere. In this case $tr(A)=1$ and $\mu^T A \mu = \mu_i^2$
  • When $i \neq j$, $A_{i,j}=A_{j,i}=1/2$ and 0 elsewhere. In this case $tr(A)=0$ and $\mu^T A \mu = \mu_i\mu_j$

Using these matrices in Eq(2), we get, for $i=j$:

$$\mathbb{E}\left[\frac{XX^T}{X^T X}\right]_{i,i} = \frac{1}{n} {}_1F_1\left(1; \frac{n}{2}+1; \frac{-||\mu||^2}{2}\right) + \frac{1}{n+2} {}_1F_1\left(1; \frac{n}{2}+2; \frac{-||\mu||^2}{2}\right) \mu_i^2$$

and for $i\neq j$: $$\mathbb{E}\left[\frac{XX^T}{X^T X}\right]_{i,j} = \frac{1}{n+2} {}_1F_1\left(1; \frac{n}{2}+2; \frac{-||\mu||^2}{2}\right) \mu_i\mu_j$$

With a little of linear algebra, we notice that these two expressions equal the first formula in this answer.

As a computational note, in case it's of use to anyone, the implementation of the confluent hypergeometric function in scipy was blowing up frequently for me. The package mpmath that allows for arbitrary precision computation has an implementation that has worked perfectly so far.

If you find this useful, please consider upvoting the answer and/or question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.