Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a set of values, each of these values has its own mean and variance. I want to be able to account for this variance when I plot the histogram of the means. Something like an error bar on the bins. Is there a commonly used method to this?

share|improve this question

2 Answers 2

up vote 4 down vote accepted

It's not clear to me why this would be an issue in a simple plot of a univariate distribution. However, it's worth knowing that histograms aren't typically considered to be a very good way to visualize a univariate distribution. Kernel density plots are generally considered to be better. These are estimated by drawing a very small distribution (typically normal) for each data point. The mean is centered on the datum in question; what's needed then is to determine the SD for the distributions (there are some fairly standard algorithms for this). Once all of the distributions have been drawn, the vertical positions of these drawn distributions are summed at each point on the horizontal axis, and that set of (x,y) pairs forms the final curve that's plotted (there's a very good figure at the linked Wikipedia page). What you could do is use your known variances for each point instead. This will most likely require a little programming on your part, but shouldn't be too hard.

share|improve this answer
    
thank you very much for your answer, summing the kernel densities will do the trick –  zamazalotta Feb 27 '13 at 2:43
    
You're welcome, @zamazalotta. If you want to code it in R, the basic procedure would be plot(density(x)) (althought that wouldn't give you the customized version). If you need help w/ the specialized programming, it's probably best to ask on Stack Overflow. –  gung Feb 27 '13 at 2:47

While I think @gung's answer is probably the right approach here, I'd like to address the original question on its own terms; I think it's an interesting question that also deserves an answer.

There are two issues here:

1) histogram bars already have an uncertainty associated with them.

2) You're apparently talking about an additional source of uncertainty (or, depending on how your means and standard deviations arise, arguably a different source for different circumstances) introduced by not being certain which bar an observation should "really" count in. We can compute that, if we make some assumptions about the distribution.

Consider an "observation", $m_i$ with standard deviation of $s_i$. I am going to treat these like known rather than estimated values.

Then, as in @gung's answer, we replace these values with kernels (of area $1/n$ if we want the histogram to estimate densities) with the same mean and standard deviation. I'm going to assume a Gaussian kernel but if you thought some other distributional assumption was more suitable, you would certainly use it instead. If you wanted to make if more efficient calculation-wise you could consider a kernel with finite range, such as an Epanechnikov.

For each little kernel, there's a certain proportion in each histogram bin. Let's say the proportion of the area of point $i$ in bin $k$ is $p_{ik}$. Then the contribution of that proportion of the point's probability to the variance of that bin is just the Bernoulli variance, $p_{ik}(1-p_{ik})$. So for bin $k$, you just sum those contributions for each point:

$$\sum_{i} p_{ik}(1-p_{ik})$$

If those $m$'s and $s$'s are estimated rather than known, it becomes more complex ... and you need to consider that first source of uncertainty.

share|improve this answer
    
+1, this is a very interesting take on the question. –  gung Apr 25 '13 at 22:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.