2
$\begingroup$

In a large class of introductory Statistics students, the professor has each student toss a coin 16 times and calculate the proportion of his or her tosses that were heads. The students then report their results, and the professor plots a Histogram of these proportions. What shape would you expect the Histogram to be? Why?

The answer says: We would expect the Histogram to be uni-modal and symmetric (i.e., shaped like a Normal curve) because the probability of heads and tails are equal.

Why does the fact that the probabilities of heads and tails are equal imply a uni-modal and symmetric Histogram?

$\endgroup$
7
  • $\begingroup$ Hi: It sounds like your professor is attempting to teach the central limit theorem but the central limit theorem doesn't necessarily hold for $n = 16$ so I'm not sure why he-she expects that. For a sample size of 35-40 or more, you would expect that because of what the CLT says. Is it possible that n was 160 instead of 16 ? $\endgroup$
    – mlofton
    Commented Jan 29, 2022 at 23:24
  • $\begingroup$ The value is given as $n = 16$. However, this book is not using the standard $n\geq 30$ as the CLT rule, they are using conditions such as $n < 15$, $15\leq n \leq 40$ and $n > 40$. Since our sample size falls in the second category, our distribution would have to be either Normal or slightly skewed with no outliers to satisfy what is called the Sample Size Assumption as part of the CLT. $\endgroup$
    – Chesso
    Commented Jan 29, 2022 at 23:28
  • $\begingroup$ @mlofton there's no need to invoke the CLT here, though. $\endgroup$
    – Glen_b
    Commented Jan 30, 2022 at 1:51
  • $\begingroup$ The CLT need not be involved, but in order for the histogram to look much like the PDF of a binomial distribution this is going to have to be a really big class. (LLN.) // See my Answer. $\endgroup$
    – BruceET
    Commented Jan 30, 2022 at 3:26
  • $\begingroup$ Thanks Ben and BruceET. For my education ( and maybe others ? ) could you elaborate a little on why CLT is not needed ? I remember that binomial goes to normal ( with some condition like p small and n large ? ) without CLT so is that what you are referring to ? If so, then i'm a little clearer but not much.. Thanks. $\endgroup$
    – mlofton
    Commented Jan 30, 2022 at 11:32

2 Answers 2

0
$\begingroup$

If there are 100 students in the class and they are tossing fair coins independently, then you have a sample of size 100 from $\mathsf{Binom}(16, 0.5).$

Simulating this 100-student experiment in R, I get the histogram shown.

set.seed(129)
x = rbinom(100, 16, .5)
cutp = (0:17)-.5
hdr = "100 observations from BINOM(16,.5)"
hist(x, prob=T, br=cutp, col="skyblue2", main=hdr)

enter image description here

The following copy of this histogram has red dots showning expected probabilities for each value in the histogram. (One hundred observations are not enough for the histogram bars to be close to their expected heights.)

k = 0:16;  pdf = dbinom(k, 16, .5)
hist(x, prob=T, br=cutp, col="skyblue2", main=hdr)
points(k, pdf, pch=19, col="red")

enter image description here

If this were a really large class of $10\,000$ online students, then (after tabulation of all the results) the figure might look like this:

set.seed(2022)
y = rbinom(10^4, 16, .5)
cutp = (0:17)-.5
hdr = "10,000 observations from BINOM(16,.5)"
hist(y, prob=T, br=cutp, col="skyblue2", main=hdr)
 points(k, pdf, pch=19, col="red")

enter image description here

$\endgroup$
1
  • 1
    $\begingroup$ BruceET; That's interesting. But what is the theory behind it ? Is that the binomial to normal approximation for large $n_2$ where $n_2$ is the number of observations drawn from the binomial ? So, then the same thing would have happened for say binomial(6, 0.5) if there were $n_2 = 10,000$ students. Essentially, there's no condition on the sample size of the binomial itself since it's large because of the 10K students ? Thanks. $\endgroup$
    – mlofton
    Commented Jan 30, 2022 at 11:45
1
$\begingroup$

Why does the fact that the probabilities of heads and tails are equal imply a uni-modal and symmetric Histogram?

First let's leave aside a histogram of some sample for a moment, and focus on the population distribution before we come back to discuss the histogram issue.

In this case we need to expand the intuitive concept of the term "unimodal" slightly, in order to count more than one adjacent-but-equal largest probabilities as constituting a single "mode" (strictly we only need it for two for the specific case at hand but the usual definition tends to be of this form).

Demonstrating this for the binomial is reasonably straightforward (it comes down to considering relationships in Pascal's triangle) but I am going to discuss the more general question - does a sum of independent symmetric unimodal variates have a distribution that's symmetric and unimodal?

The answer is yes in the general case, for both continuous and discrete variables.

Symmetry is fairly straightforward.

When we add two independent random variables, the pmf (/density) of the sum is the convolution of the two pmfs (/densities).

That is, $P(X_1 + X_2 = t) = \sum_k P(X_1=k) \times P(X_2 = t-k)$ where the sum is over the possible values of $k$ (there's an equivalent statement for continuous functions but let's stick with the discrete one right now).

[Two asides: (i) the convolution of two unimodal distributions is not always unimodal (consider probabilities of $\{\frac46,\frac16,\frac16\}$ on $\{0,1,2\}$), so we have to be careful to keep to the symmetric case; (ii) we can demonstrate the general case of convolution of n variables by considering convolutions of just two variables, since the convolution of three variables can be written as the convolution of the first two, itself convolved with the third, and so forth for more variables.]

If the two input functions are symmetric about some centers ($m_1$ and $m_2$ respectively), then the output of the convolution will be symmetric about $m=m_1+m_2$.

In the sum for $t=m+s$ and $t=m-s$ there's a corresponding term equally far above / below $m_1$ and $m_2$ in each sum (using the symmetry of the input functions) so that we can pair up every term in each product within each sum with one that it is equal to.

Symmetric and Unimodal

There is a proof in [2] which relies on two steps to establish the case for discrete distributions on the integers. It restricts attention to the case where the centers of symmetry are at $0$ without loss of generality.

The first step establishes that a unimodal distribution on integers can be written uniquely as a mixture of uniforms. (Theorem 4.3, which I will not attempt to reproduce). The second (Theorem 4.7) shows that convolutions of uniforms are unimodal (which is straightforward) and uses the fact that the convolution of mixtures of uniforms is a mixture of convolutions of uniforms.

There's a proof in [1] which relies on the fact that if $X_1,X_2$ are identically distributed independent variates with unimodal distributions then $X_1-X_2$ is symmetric, a result originally due to Hodges and Lehmann. The paper contains a proof that convolutions of symmetric unimodal densities are symmetric unimodal for the continuous case and mentions the methods extend to the discrete case.

(I presume other proofs exist outside the statistics literature but I haven't seen any.)

Histograms

A histogram is best suited to displaying a sample from a continuous variable (or a variable with such a huge number of discrete outcomes that displaying the individual proportions would give a visually cluttered display -- probably well into the hundreds of possible outcomes for a discrete distribution with equally-spaced outcomes).

Consequently I would not use a histogram to display such a sample, at least not without a great deal of care; rather if I wanted to show an empirical pmf I would instead simply display each of the individual proportions. (For some purposes, though, I'd probably prefer instead to look at an empirical cdf; it depends on what we're looking for.)

If you must use a histogram rather than directly plot the outcomes you should try hard to make it that the bins are of the same width as the distance between adjacent possible outcomes (or some simple fraction of that distance) and that the bin-centers are aligned with the possible outcomes. This avoids the misleading possibility of combining some outcomes into a single bin in an uneven way.

BruceET does this in his answer, so his histograms are not misleading in the way I was warning against. The red points he displays are the kind of display I was advocating as a good way to show pmfs (population or sample).

I agree with his warning about judging the symmetry of populations from the symmetry of samples. In general samples will be asymmetric and an appearance of near symmetry is not the same thing as having symmetry in the population.


[1] Purkayastha, S., 1998.
"Simple proofs of two results on convolutions of unimodal distributions".
Statistics & Probability Letters, Volume 39, Issue 2, p 97-100.

[2] Dharmadhikari, S.W., Joag-Dev, K., 1988.
Unimodality, Convexity, and Applications.
Academic Press, New York.

$\endgroup$
1
  • $\begingroup$ Thanks for your detailed and useful response. I get the idea a lot better now and the references are great if I ever want to look at the gory details. Also, your point is consistent with BruceET's plots because the symmetry is there in his plots even for $n = 100$. Great stuff. I think BruceET's 10K result where the dots fall exactly on the histogram edges is asymptotic and not the CLT but I'll worry about that some other time. $\endgroup$
    – mlofton
    Commented Jan 31, 2022 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.