6
$\begingroup$

If $X$ is a real-valued random variable,

$$\mathbb{E}[X^2] - (\mathbb{E}[X])^2$$

is the variance of $X$.

Suppose now that $X$ is a random variable that takes values on $\mathbb{R}^n$. Consider the quantity

$$f(x) = \mathbb{E}[\|X\|^2] - \|\mathbb{E}[X]\|^2,$$

where $\|x\|$ is the $L_2$ norm of the vector $x$: i.e., $\|x\|^2 = x^\top x = x_1^2 + \dots + x_n^2$, and where $\mathbb{E}[X]$ is the coordinate-wise expected value of $X$, i.e., it is a $n$-vector whose $i$th component is $\mathbb{E}[X_i]$. This looks like some kind of generalization of the variance to higher dimensions, where we replace "squaring a real number" with "the squared L_2 norm of a vector".

Is there some way to understand what the quantity $f(X)$ represents, or to get an intuition for what it is calculating? Is it some generalization of the variance?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
5
$\begingroup$

It's just the sum of the variances of each component.

Suppose $n=2$ and $X=(X_1,X_2)$. Then

$$\mathbb{E}[\|X\|^2] = \mathbb{E}[X_1^2 + X_2^2] = \mathbb{E}[X_1^2] + \mathbb{E}[X_2^2].$$

Also

$$\|\mathbb{E}[X]\|^2 = \|(\mathbb{E}[X_1],\mathbb{E}[X_2])\|^2 = \mathbb{E}[X_1]^2 + \mathbb{E}[X_2]^2.$$

Therefore

$$\begin{align*} \mathbb{E}[\|X|^2] - \|\mathbb{E}[X]\|^2 &= \left( \mathbb{E}[X_1^2] - \mathbb{E}[X_1]^2 \right) + \left( \mathbb{E}[X_2^2] - \mathbb{E}[X_2]^2 \right) \\ &= \text{Var}(X_1) + \text{Var}(X_2).\end{align*}$$

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ I don't see why ||(E($X_1,$$ X_2$)||$^2$ =E[$X_1$]$^2$ + E[$X_2]$$^2$. $\endgroup$
    – Michael R. Chernick
    Commented May 20, 2017 at 1:39
  • $\begingroup$ @MichaelChernick, it's because $\|(u,v)\|^2 = u^2 + v^2$ (by definition of the squared-norm). Now plug in $u=\mathbb{E}[X_1]$ and $v=\mathbb{E}[X_2]$. (Or did I go wrong somewhere?) $\endgroup$
    – D.W.
    Commented May 20, 2017 at 4:30
  • 1
    $\begingroup$ Suppose $X$ can take the values $(1,1)$ or $(-1,-1)$ each with probability $\frac12$. I would have thought $\|X\| = 2$ with probability $1$ and so $\mathbb{E}[\|X\|^2] = 4$, while $\mathbb{E}[X] =(0,0)$ so $\|\mathbb{E}[X]\|^2=0$ making the difference $4-0=4$. But the sum of the variances is $1+1=2$ $\endgroup$
    – Henry
    Commented May 20, 2017 at 6:50
  • 1
    $\begingroup$ @Henry, no, in your example $\|X\|=\sqrt{2}$ and $\|X\|^2 = 2$. Remember, $\|\cdot\|$ is the $L_2$ norm, and $\|\cdot\|^2$ is the squared $L_2$ norm, given by $\|x\|^2 = x_1^2 + \dots + x_n^2$, or in the case of $n=2$ dimensions, $\|x\|^2 = x_1^2 + x_2^2$. Therefore $\|X\|^2=2$ and $\mathbb{E}[\|X\|^2] = 2$, making the difference 2 -- which is equal to the sum of the variances. $\endgroup$
    – D.W.
    Commented May 20, 2017 at 11:11
  • 1
    $\begingroup$ I agree with D.W. answer now and think he can say it is related to variance in the sense that it is the sum of variances. $\endgroup$
    – Michael R. Chernick
    Commented May 20, 2017 at 15:56
0
$\begingroup$

I do not think $f(x)$ has any meaning, and absolutely it is not generalization of variance. The generalization of variance is variance-covaviance matrix defined by

$E(XX')-E(X)E(X)'$ where $X$ is random column vector.

Improve this answer
$\endgroup$
6
  • 4
    $\begingroup$ If $X$ is random vector then $\Sigma = \operatorname{E}(XX')-\operatorname{E}(X)\operatorname{E}(X)'$ is indeed the covariance matrix and generalizes the notion of variance to random vectors. What the OP appears to be interested in is $\operatorname{E}(X'X)-\operatorname{E}(X)'\operatorname{E}(X)$ which is the trace of $\Sigma$. $\endgroup$
    – Matthew Gunn
    Commented May 20, 2017 at 7:39
  • 1
    $\begingroup$ Absolutely agree with @Matthew Gunn. It does have a meaning and it is interpretable as a measure of 'total dispersion' (i.e. sum of the eigenvalues of the covariance matrix), which to my mind is a generalisation of variance. Of course the covariance matrix contains more information, but the familiar one-dimensional variance is a special case of OPs 'variance', hence it's a generalisation. $\endgroup$
    – Will
    Commented May 20, 2017 at 12:36
  • 1
    $\begingroup$ The trace is equal to the sum of the eigenvalues, isn't it? I'm not denying that the covariance matrix is more informative than $f$. I'm just saying that it is a measure of dispersion and it has the 1D variance as a special case. Why does the mean exist when we can just look at the whole distribution? Because sometimes that's all we need. $\endgroup$
    – Will
    Commented May 20, 2017 at 13:34
  • 2
    $\begingroup$ Yes, I am wrong, trace = sum of eigenvalue. $\endgroup$
    – user158565
    Commented May 20, 2017 at 13:42
  • 1
    $\begingroup$ But I still think $f(x)$ will mislead the people when $X$ are not independent if we consider it as the generalization of variance. If you insist on one number to reflect the variation of $X$, I prefer the $Var(\sum(X))$ over $f(x)$. $\endgroup$
    – user158565
    Commented May 20, 2017 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.