2
$\begingroup$

I have a simulation data for a random variable $X$, and also a parameter $p$. I am plotting the average $\langle X\rangle$ vs $p$. Now, I also want to show error bars on the plot to show the spread of $X$. When I plot the histogram of $X$, it looks far from Gaussian (it isn't even symmetric), and so I think that symmetric error bars using standard deviation $\sigma_X$ doesn't make much sense. For example, $\langle X\rangle -\sigma_X$ sometimes goes below $0$, whereas $X$ is known to take values in $[0, 1]$. I think that percentiles would make more meaningful error bars here because the distribution is not symmetric. However, what percentage would be a better option option for error bars: $68, 95, 99$? Also, since an error bar's lengths on the two sides of the average are going to be unequal, how would I choose them ($95\%$ on each side or $95\%$ in total or what)? Thanks in advance.

$\endgroup$
6
  • 2
    $\begingroup$ By 95% confidence, it is implied that the value falls within such a range 95% of the time, so that would correspond to what you call 95% in total. As for your question. Is there a problem with calculating asymmetric bars? If so, could you edit your question to clarify what it is you have trouble with? $\endgroup$ Jul 1, 2019 at 6:31
  • $\begingroup$ I do have a technical problem in calculating asymmetric error bars, but I think that is a different problem. Here I am simply asking what percentage of values should fall within the range shown by error bars? Or in other words, what is the standard? $\endgroup$
    – Peaceful
    Jul 1, 2019 at 6:38
  • 2
    $\begingroup$ If anything, the standard is 95%, but you are completely free to use your own value. In fact, what matters more than the value you choose is that you clearly indicate the value you used. $\endgroup$ Jul 1, 2019 at 6:53
  • $\begingroup$ What is the interval shown by the error bars intended to represent? $\endgroup$
    – Glen_b
    Jul 1, 2019 at 10:22
  • $\begingroup$ @Glen_b: The spread of the data, not the accuracy with which the mean was estimated. $\endgroup$
    – Peaceful
    Jul 1, 2019 at 10:32

2 Answers 2

4
$\begingroup$

The short answer is - yes, you can (and in my opinion, should) just take the middle 95% percentile values (i.e. from 2.5% percentile to 97.5% percentile) as your error bars.
The choice of 95% (2.5%-97.5%) is of course subjective, and you should use whatever's acceptable in your context - 95% is kind of a standard, but personally I've used and seen 99%, 90%, and even 80% where the distribution is very wide.

$\endgroup$
3
  • 1
    $\begingroup$ The important point is to clearly spell out what the bars mean. There's nothing wrong with plotting/reporting mean (or median) and 2.5 th to 97.5th percentiles of observed data points. Nor with the shortest interval covering 95 % of the observed data points. $\endgroup$ Jul 1, 2019 at 11:51
  • $\begingroup$ @cbeleites do you know of a fast way to look for the shortest interval covering 95% of observed data points? I don't and would like to $\endgroup$ Jul 1, 2019 at 13:23
  • 2
    $\begingroup$ so far the iterative procedure has been fine for me: sort data points, in vectorized programming language calculate diff vector. Start at the mode. Expand to closest data point. Repeat until deserved coverage: number of iteration steps can be calculated beforehand (95 % of known no. of data points). You can also turn around the procedure and shrink the interval always the side with larger difference aes that will need less steps. If iterations are costly, it may be faster to start with 2.5th to 97.5th interval and move that to shorten. $\endgroup$ Jul 1, 2019 at 13:37
4
$\begingroup$

I think I detect some confusion about what error bars may represent.

  • "Error bars" can indicate the spread of an observed distribution, typically by extending over certain quantiles, e.g., going from the 2.5% to the 97.5% quantile.

    This is what Itamar seems to implicitly assume you want.

    However, note that there is no "error" involved here, so "error bars" is a misnomer. Which is because the second use of error bars is much more common:

  • Error bars can be used to indicate the standard error of an estimate, e.g., the standard error of the mean. This is typically the correct interpretation of the error bars on "dynamite plots" I see (in psychology).

And note that the sampling distribution of many estimates is often symmetric, even if the underlying distribution is not, by the CLT. In which case you get a symmetric for your parameter by multiplying the SE by a $z$ value, and your concern about asymmetry simply disappears. If, that is, this is what you are actually looking for.

As an example, I'll generate 50 gamma distributed data points. The distribution is nicely asymmetric. Here is a histogram:

histogram

Now, let's plot the mean with two different error bars per above. Note how the quantile is of course much larger than the SEM, and how the quantile is asymmetric. Also note two visualizations I personally would much prefer: a beanplot (also known as a violin plot, where I personally like to add the actual observations) and a rugplot at the bottom.

beanplot etc.

R code:

set.seed(1)
nn <- 50
xx <- rgamma(nn,1,1)
xlim <- c(floor(min(xx)),ceiling(max(xx)))

library(beanplot)

hist(xx,col="grey",xlab="",main="",xlim=xlim)

opar <- par(mai=c(.5,1,.1,.1),lwd=2)
    plot(rep(mean(xx),2),2:3,pch=19,yaxt="n",ylab="",xlab="",main="",xlim=xlim,ylim=c(.5,3.5),cex=1.2)
    lines(quantile(xx,c(.025,.975)),rep(3,2))
    lines(mean(xx)+c(-1,1)*sd(xx)/sqrt(length(xx)),rep(2,2))
    beanplot(xx,horizontal=TRUE,what=c(0,1,0,0),xlim=xlim,col="grey",add=TRUE,at=1)
    points(xx,runif(nn,.8,1.2),pch=19,cex=0.6)
    rug(xx)
    axis(2,1:3,c("Beanplot","Mean &\nSEM","Mean &\nquantile"),las=1)
par(opar)
$\endgroup$
2
  • 1
    $\begingroup$ Your explanation is very good all around (from the start to the visualization), but I think OP was pretty explicit with their use-case - they want to "show error bars on the plot to show the spread of X". $\endgroup$ Jul 1, 2019 at 10:21
  • 1
    $\begingroup$ Also, note that OP talks about a simulation (getting X as a function of p), so showing the simulated data points is not (in my humble opinion) as important as it is when presenting and analyzing empirical data points. $\endgroup$ Jul 1, 2019 at 10:23

Not the answer you're looking for? Browse other questions tagged or ask your own question.