Suppose I sample $n$ times from a distribution $$ x_1, \ldots, x_n \sim p_\theta(x) $$ is the mean of the samples always a valid sample from the target distribution? I.e. is $\overline{x}$ a valid sample from $p_\theta(x)$ $$ \overline{x} = \frac{1}{n}\sum_{i=1}^n x_i $$
$\begingroup$
$\endgroup$
5
asked Apr 30, 2020 at 10:43
-
2$\begingroup$ My intuition is that it's not. For instance, suppose the distribution is multimodal $\endgroup$– Euler_SalterCommented Apr 30, 2020 at 10:48
-
3$\begingroup$ You may want to read about the Central Limit Theorem. $\endgroup$– PatrickTCommented May 1, 2020 at 7:03
-
1$\begingroup$ Is the question whether the average has the same distribution, or whether the average is a possible sample of the distribution? $\endgroup$– Jorge LeitaoCommented May 2, 2020 at 4:49
-
$\begingroup$ What's a "valid sample"? For example, are you asking if the mean of samples from a distribution is necessary a member of the set of values that the distribution includes? $\endgroup$– NatCommented May 3, 2020 at 0:28
-
1$\begingroup$ Great answers below. Perhaps the easiest example to think of is that of rolling a die. The mean outcome is 3.5, which will never come up as an individual observation. $\endgroup$– IoannisCommented May 3, 2020 at 17:26
Add a comment
|
7 Answers
$\begingroup$
$\endgroup$
5
No, $\bar x$ has its own sampling distribution. Take, for example, the variances of $\bar x$ and $x_i$, in which the former is always lower ($\leq$) than the latter, which means $\bar x$ is not sampled from $p_\theta(x)$.
-
$\begingroup$ Good point! That's such a simple thing to check $\endgroup$ Commented Apr 30, 2020 at 10:48
-
3$\begingroup$ It is a sample from $p$ only when $n=1$ btw. $\endgroup$– gunesCommented Apr 30, 2020 at 10:50
-
$\begingroup$ If I recall, the exceptional case here is the Cauchy distribution. i.e. if $x_i$ are sampled from a Cauchy distribution, then $\bar x$ is distributed as the same Cauchy distribution. $\endgroup$– kdbanmanCommented May 6, 2020 at 1:19
-
$\begingroup$ @kdbanman correct, it's also referred in other answer by javierozcoiti. Another trivial exception is constant RV. $\endgroup$– gunesCommented May 6, 2020 at 6:15
-
1$\begingroup$ Ah, that’s what I get for not reading further! @gunes thanks for the additional case. In a way, it’s interesting that the limiting cases are at the extrema of of dispersion: minimal dispersion for constant RV (i.e. delta distribution) and maximal dispersion for Cauchy. $\endgroup$– kdbanmanCommented May 6, 2020 at 18:17
$\begingroup$
$\endgroup$
Good examples so far but consider $$X_i \sim Bernoulli(.5)$$
In that case the distribution of the data will only have support on 0 and 1. But the sample mean will have an ever decreasing probability of taking a value of 0 or 1 as the sample size gets larger and larger. That alone should show that the mean isn't being sampled from the original distribution.
$\begingroup$
$\endgroup$
No. Suppose you have $X_1, X_2 \sim N(0,1)$. Then, $$ \bar{X} = \dfrac{X_1 + X_2}{2} \sim N\left(0, \dfrac{1}{2} \right)\,. $$
But $N(0,1) \ne N(0, 1/2)$.
answered Apr 30, 2020 at 10:48
$\begingroup$
$\endgroup$
3
No, it is only valid in cases as the Cauchy distribution, the means of samples of the Cauchy follow the same Cauchy dstribution.
-
1$\begingroup$ True. And it's not even the only example but since the question specifically says 'always' - examples aren't nearly as important as counter examples here. $\endgroup$– DasonCommented May 1, 2020 at 1:47
-
2$\begingroup$ (+1) This could be the perfect non-trivial answer for "can it be" type of question. $\endgroup$– gunesCommented May 1, 2020 at 11:54
-
2$\begingroup$ I would think the "perfect" can it be example would be a distribution that is a point mass. It's a lot less complicated than dealing with a Cauchy. $\endgroup$– DasonCommented May 1, 2020 at 20:03
$\begingroup$
$\endgroup$
As an even more pathological example, consider a sample from the distribution which is uniform on the union of $[0,1]$ and $[3,4]$. As the sample size increases, the mean will tend to 2 which isn't even in the support of the distribution. Another similar example is the uniform distribution on the boundary of the unit sphere (in any number of dimensions)
$\begingroup$
$\endgroup$
No.
For the average to be a sample of the distribution, it must belong to the support of the distribution.
Below are two examples where that is not the case (which is sufficient to show that the statement is not true in general).
Discrete
The distribution p(x=1) = 0.5; p(x=-1) = 0.5 has support $$S=\{-1,1\}$$ but average $0\notin S$.
Continuous
The density function
$$p(x) = \frac{1}{2}rect(x-1) + \frac{1}{2}rect(x+1)$$
(two Rectangular functions centered at 1 and -1 respectively) has support
$$S = ]-1.5,-0.5[\cap]0.5,1.5[$$
but average $0\notin S$.
$\begingroup$
$\endgroup$
It depends on $n$. In the case where $n = 1$, the sample mean $\bar{x}=x_1$ and trivially matches $p_\theta(x)$ as it is just the single sample.
For specific degenerate distributions, the mean can still match the target distribution, but this is generally not true. As yet another proof, a sample from the distribution $p_\theta(x)$ must conform to the probabilities defined by $p_\theta(x)$. So, if $p_\theta(x)$ is discrete, "valid samples" must belong to the discrete set of possible values. If $p_\theta(x)$ is continuous, valid samples must respect the density function. The mean of samples $\bar{x}$ does not necessarily satisfy these properties. Take for example the case where $p_\theta(x)$ is a Bernoulli distribution:
$$x \sim Bernoulli(\theta)$$
where $x \in \{0,1\}.$
If you sample $n=2$ and the realized samples are $x_1=0$, and $X_2=1$, the mean is $1\over{2}$, but $1\over{2}$ is not discrete and $1\over{2}$ $\notin\{0,1\}$.
