7
$\begingroup$

Let $X_1, X_2, \dotsc, X_k$ be an i.i.d. sample of a random variable $X$. I plot these in a histogram and would like to include a confidence interval for the height of each histogram bar. Do you know how to go about doing it?

$\endgroup$
  • 3
    $\begingroup$ How were the bins chosen? It's straightforward to obtain CIs for the heights when the cutpoints are chosen independently of the data, but otherwise it's much more challenging. Normally, bins for histograms are chosen on the basis of data characteristics, including their amount, their range, whether there are apparent outliers, and how skew they might be: this creates a lack of independence which is difficult to quantify (and therefore it ought to color our interpretation of any CIs we might attempt to calculate for the heights). You can avoid these complications by using a probability plot. $\endgroup$ – whuber May 25 '12 at 4:28
5
$\begingroup$

Let there be $i \in 1, \dots, I$ histogram bins. The probability of falling into a particular bin is $p_i$. This is just a binomial trial (i.e., you are either in the bin or not, each with a given probability).

If you are calculating the frequency of being in the bin (i.e., histograms with bars giving the set of $p_i$'s), then the variance should be $p_i(1-p_i)/k$.

If you are calculating total counts, then the variance is $p_i(1-p_i) \times k$.

The confidence interval can then be formed in the standard way.

$\endgroup$
  • $\begingroup$ Let say I have 100,000 samples. 30,000 of them fall into bin 1 (e.g. values 0-100). Then p_1 = 0.3 and variance = p_1*(1-p_1)/100,000 = 0.3*0.7/100,000? $\endgroup$ – John Manak May 25 '12 at 2:24
  • $\begingroup$ @JohnManak, Yes, that's right. $\endgroup$ – Charlie May 25 '12 at 2:26
  • 1
    $\begingroup$ Shouldn't the variance rather be 100,000*0.3*0.7? $\endgroup$ – John Manak May 25 '12 at 2:29
  • $\begingroup$ Ahh, good point @JohnManak, I usually plot frequencies as in proportions. I'll make it clearer. $\endgroup$ – Charlie May 25 '12 at 2:35
5
$\begingroup$

Using the binomial variance $p (1-p) k$, as proposed in another answer, is only a good idea when the proportion is not near 0 or 1.

For better-behaved confidence intervals, there is extensive statistical literature e.g. Agresti & Coull (1998). Some of the proposed formulae are implemented in the R library PropCIs. Here's an example creating a histogram with error bars using PropCIs: Errorbars on histograms

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.