12
$\begingroup$

Suppose I sample $n$ times from a distribution $$ x_1, \ldots, x_n \sim p_\theta(x) $$ is the mean of the samples always a valid sample from the target distribution? I.e. is $\overline{x}$ a valid sample from $p_\theta(x)$ $$ \overline{x} = \frac{1}{n}\sum_{i=1}^n x_i $$

$\endgroup$
5
  • 2
    $\begingroup$ My intuition is that it's not. For instance, suppose the distribution is multimodal $\endgroup$ Apr 30 '20 at 10:48
  • 3
    $\begingroup$ You may want to read about the Central Limit Theorem. $\endgroup$
    – PatrickT
    May 1 '20 at 7:03
  • 1
    $\begingroup$ Is the question whether the average has the same distribution, or whether the average is a possible sample of the distribution? $\endgroup$ May 2 '20 at 4:49
  • $\begingroup$ What's a "valid sample"? For example, are you asking if the mean of samples from a distribution is necessary a member of the set of values that the distribution includes? $\endgroup$
    – Nat
    May 3 '20 at 0:28
  • $\begingroup$ Great answers below. Perhaps the easiest example to think of is that of rolling a die. The mean outcome is 3.5, which will never come up as an individual observation. $\endgroup$
    – Ioannis
    May 3 '20 at 17:26
22
$\begingroup$

No, $\bar x$ has its own sampling distribution. Take, for example, the variances of $\bar x$ and $x_i$, in which the former is always lower ($\leq$) than the latter, which means $\bar x$ is not sampled from $p_\theta(x)$.

$\endgroup$
5
  • $\begingroup$ Good point! That's such a simple thing to check $\endgroup$ Apr 30 '20 at 10:48
  • 3
    $\begingroup$ It is a sample from $p$ only when $n=1$ btw. $\endgroup$
    – gunes
    Apr 30 '20 at 10:50
  • $\begingroup$ If I recall, the exceptional case here is the Cauchy distribution. i.e. if $x_i$ are sampled from a Cauchy distribution, then $\bar x$ is distributed as the same Cauchy distribution. $\endgroup$
    – kdbanman
    May 6 '20 at 1:19
  • $\begingroup$ @kdbanman correct, it's also referred in other answer by javierozcoiti. Another trivial exception is constant RV. $\endgroup$
    – gunes
    May 6 '20 at 6:15
  • 1
    $\begingroup$ Ah, that’s what I get for not reading further! @gunes thanks for the additional case. In a way, it’s interesting that the limiting cases are at the extrema of of dispersion: minimal dispersion for constant RV (i.e. delta distribution) and maximal dispersion for Cauchy. $\endgroup$
    – kdbanman
    May 6 '20 at 18:17
21
$\begingroup$

Good examples so far but consider $$X_i \sim Bernoulli(.5)$$

In that case the distribution of the data will only have support on 0 and 1. But the sample mean will have an ever decreasing probability of taking a value of 0 or 1 as the sample size gets larger and larger. That alone should show that the mean isn't being sampled from the original distribution.

$\endgroup$
13
$\begingroup$

No, it is only valid in cases as the Cauchy distribution, the means of samples of the Cauchy follow the same Cauchy dstribution.

$\endgroup$
3
  • $\begingroup$ True. And it's not even the only example but since the question specifically says 'always' - examples aren't nearly as important as counter examples here. $\endgroup$
    – Dason
    May 1 '20 at 1:47
  • 2
    $\begingroup$ (+1) This could be the perfect non-trivial answer for "can it be" type of question. $\endgroup$
    – gunes
    May 1 '20 at 11:54
  • 2
    $\begingroup$ I would think the "perfect" can it be example would be a distribution that is a point mass. It's a lot less complicated than dealing with a Cauchy. $\endgroup$
    – Dason
    May 1 '20 at 20:03
12
$\begingroup$

As an even more pathological example, consider a sample from the distribution which is uniform on the union of $[0,1]$ and $[3,4]$. As the sample size increases, the mean will tend to 2 which isn't even in the support of the distribution. Another similar example is the uniform distribution on the boundary of the unit sphere (in any number of dimensions)

$\endgroup$
9
$\begingroup$

No. Suppose you have $X_1, X_2 \sim N(0,1)$. Then, $$ \bar{X} = \dfrac{X_1 + X_2}{2} \sim N\left(0, \dfrac{1}{2} \right)\,. $$

But $N(0,1) \ne N(0, 1/2)$.

$\endgroup$
1
$\begingroup$

No.

For the average to be a sample of the distribution, it must belong to the support of the distribution.

Below are two examples where that is not the case (which is sufficient to show that the statement is not true in general).

Discrete

The distribution p(x=1) = 0.5; p(x=-1) = 0.5 has support $$S=\{-1,1\}$$ but average $0\notin S$.

Continuous

The density function

$$p(x) = \frac{1}{2}rect(x-1) + \frac{1}{2}rect(x+1)$$

(two Rectangular functions centered at 1 and -1 respectively) has support

$$S = ]-1.5,-0.5[\cap]0.5,1.5[$$

but average $0\notin S$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.