12
$\begingroup$

Suppose I sample $n$ times from a distribution $$ x_1, \ldots, x_n \sim p_\theta(x) $$ is the mean of the samples always a valid sample from the target distribution? I.e. is $\overline{x}$ a valid sample from $p_\theta(x)$ $$ \overline{x} = \frac{1}{n}\sum_{i=1}^n x_i $$

$\endgroup$
5
  • 2
    $\begingroup$ My intuition is that it's not. For instance, suppose the distribution is multimodal $\endgroup$ – Euler_Salter Apr 30 '20 at 10:48
  • 3
    $\begingroup$ You may want to read about the Central Limit Theorem. $\endgroup$ – PatrickT May 1 '20 at 7:03
  • 1
    $\begingroup$ Is the question whether the average has the same distribution, or whether the average is a possible sample of the distribution? $\endgroup$ – Jorge Leitao May 2 '20 at 4:49
  • $\begingroup$ What's a "valid sample"? For example, are you asking if the mean of samples from a distribution is necessary a member of the set of values that the distribution includes? $\endgroup$ – Nat May 3 '20 at 0:28
  • $\begingroup$ Great answers below. Perhaps the easiest example to think of is that of rolling a die. The mean outcome is 3.5, which will never come up as an individual observation. $\endgroup$ – Ioannis May 3 '20 at 17:26
22
$\begingroup$

No, $\bar x$ has its own sampling distribution. Take, for example, the variances of $\bar x$ and $x_i$, in which the former is always lower ($\leq$) than the latter, which means $\bar x$ is not sampled from $p_\theta(x)$.

$\endgroup$
5
  • $\begingroup$ Good point! That's such a simple thing to check $\endgroup$ – Euler_Salter Apr 30 '20 at 10:48
  • 3
    $\begingroup$ It is a sample from $p$ only when $n=1$ btw. $\endgroup$ – gunes Apr 30 '20 at 10:50
  • $\begingroup$ If I recall, the exceptional case here is the Cauchy distribution. i.e. if $x_i$ are sampled from a Cauchy distribution, then $\bar x$ is distributed as the same Cauchy distribution. $\endgroup$ – kdbanman May 6 '20 at 1:19
  • $\begingroup$ @kdbanman correct, it's also referred in other answer by javierozcoiti. Another trivial exception is constant RV. $\endgroup$ – gunes May 6 '20 at 6:15
  • 1
    $\begingroup$ Ah, that’s what I get for not reading further! @gunes thanks for the additional case. In a way, it’s interesting that the limiting cases are at the extrema of of dispersion: minimal dispersion for constant RV (i.e. delta distribution) and maximal dispersion for Cauchy. $\endgroup$ – kdbanman May 6 '20 at 18:17
21
$\begingroup$

Good examples so far but consider $$X_i \sim Bernoulli(.5)$$

In that case the distribution of the data will only have support on 0 and 1. But the sample mean will have an ever decreasing probability of taking a value of 0 or 1 as the sample size gets larger and larger. That alone should show that the mean isn't being sampled from the original distribution.

$\endgroup$
13
$\begingroup$

No, it is only valid in cases as the Cauchy distribution, the means of samples of the Cauchy follow the same Cauchy dstribution.

$\endgroup$
3
  • $\begingroup$ True. And it's not even the only example but since the question specifically says 'always' - examples aren't nearly as important as counter examples here. $\endgroup$ – Dason May 1 '20 at 1:47
  • 2
    $\begingroup$ (+1) This could be the perfect non-trivial answer for "can it be" type of question. $\endgroup$ – gunes May 1 '20 at 11:54
  • 2
    $\begingroup$ I would think the "perfect" can it be example would be a distribution that is a point mass. It's a lot less complicated than dealing with a Cauchy. $\endgroup$ – Dason May 1 '20 at 20:03
12
$\begingroup$

As an even more pathological example, consider a sample from the distribution which is uniform on the union of $[0,1]$ and $[3,4]$. As the sample size increases, the mean will tend to 2 which isn't even in the support of the distribution. Another similar example is the uniform distribution on the boundary of the unit sphere (in any number of dimensions)

$\endgroup$
9
$\begingroup$

No. Suppose you have $X_1, X_2 \sim N(0,1)$. Then, $$ \bar{X} = \dfrac{X_1 + X_2}{2} \sim N\left(0, \dfrac{1}{2} \right)\,. $$

But $N(0,1) \ne N(0, 1/2)$.

$\endgroup$
1
$\begingroup$

No.

For the average to be a sample of the distribution, it must belong to the support of the distribution.

Below are two examples where that is not the case (which is sufficient to show that the statement is not true in general).

Discrete

The distribution p(x=1) = 0.5; p(x=-1) = 0.5 has support $$S=\{-1,1\}$$ but average $0\notin S$.

Continuous

The density function

$$p(x) = \frac{1}{2}rect(x-1) + \frac{1}{2}rect(x+1)$$

(two Rectangular functions centered at 1 and -1 respectively) has support

$$S = ]-1.5,-0.5[\cap]0.5,1.5[$$

but average $0\notin S$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.