Error bars in a population and subtracting two populations with different error bars

Suppose I want to measure a physical quantity. Let's say that $$N$$ trials were performed each with individual outcome $$x_i,\quad i\in (1,N)$$. Then obviously the outcome of the experiment would be the mean $$m=\frac{1}{N}\sum_{i=1}^Nx_i$$. But, what about the error bars involved? I think the result should be written as $$(m\pm \Delta m)$$ wherein $$\Delta m= max(|x_i-m|)$$ so that it definitely incorporates the confidence interval of the population. Am I right, or am I wrong? Kindly help knowing that I have no background in statistics.

The reason I want to use the error bar term is that, I want to measure the step height of a material, whose line profile looks like the image shown.

As you can see the floor (circled in light red) has several data points, the first step, marked in dark red, also has several data points, while the second step, marked in yellow, also has several data points. I want to be able to say that in this graph my floor exists at a height of $$(f\pm\Delta f)$$ while my first step exists at a height of $$(s_1\pm\Delta s_1)$$ and my second step at a height of $$(s_2\pm\Delta s_2)$$. Then I would want to subtract the position of my floor from the first step (along with the error bars) and say, relative to the floor, the height of my first step is $$(h_1\pm\Delta h_1)$$ and the height of the second step is $$(h_2\pm\Delta h_2)$$. (How to subtract two populations with different error bars would have been my next question in this site, but obviously we know now that it does not work that way...lol.)

• An error bar can mean whatever you want it to mean, but it still has to make some sense, and the computation has to correspond with your interpretation. So the question is: what do you want your errorbar to mean and how does that correspond with your computation? Your computation is very unusual (overly conservative), so your interpretation should also be unusual. Jan 17 '20 at 15:45
• @MaartenBuis Could you let me know a bit more about what you mean when you say that my computation is conservative and that the error bar can mean whatever I want it to mean?? Jan 17 '20 at 16:02
• A basic--and perhaps surprising--result of statistical theory is that the maximum distance between the mean of a bunch of measurements and the measurements themselves is not a confidence interval for the mean. Thus, we ought to take a step back and ask you what your aim is, and that's what @Maarten is asking: what does an "error bar" mean to you? What properties should it have, how will you interpret it, and how will you use it for future analyses or decisions?
– whuber
Jan 17 '20 at 17:48
• To add to @whuber, who captured most of what I wanted to say, statistical terminology is not as standardized as many think. Error bar is an example of such a term that: it could mean a confidence interval, but if you specify a different definition then that is fine too. You obviously need to make sure that the audience knows your definition. So, since your computation was clearly not a confidence interval, I assumed you wanted something else. Jan 18 '20 at 11:30
• Telling what you want if you don't know the terminology is obviously hard. Maybe the easiest way to communicate what you want is to tell us why you want it. What is that bar supposed to tell the audience? It is usually much easier for us to reconstruct what you want from that information. Jan 18 '20 at 11:33

If I read your question correctly, you want to infer the distances between the plateau heights and the associated uncertainty of this inference. If it makes sense to think of this as a change point model where the location of the changes are of interest too, the mcp package may be of some use. It is a package for Bayesian change-point regression using MCMC sampling of the (joint) posteriors. The nice thing about such samples is that you can add and subtract to your liking while still correctly representing the uncertainty.

So say you just have three plateaus:

df = data.frame(width = seq(20, 35-0.1, 0.1),
height = c(rnorm(2/0.1, 2.5, 0.1),
rnorm(3/0.1, 2.0, 0.1),
rnorm(10/0.1, 0.3, 0.1)))

Now let's model this as three plateau segments:

model = list(
height ~ 1,
~ 1,
~ 1
)

We fit it and summarise:

library(mcp)
fit = mcp(model, data = df, par_x = "width")
plot(fit) At this point, you can inspect the overall fit and individual parameters using summary(fit) and plot(fit) and plot_pars(fit). But for testing specific differences, you can use hypothesis to add/subtract samples and get some inferential tests too:

> hypothesis(fit, c("int_2 - int_3 > 0", "int_1 - int_3 > 0", "int_1 - int_2 > 0"))
#          hypothesis      mean     lower    upper         p       BF
# 1 int_2 - int_3 > 0 1.6038992 1.6211943 1.724851 0.9836667 60.22449
# 2 int_1 - int_3 > 0 2.1916182 2.1363059 2.265295 1.0000000      Inf
# 3 int_1 - int_2 > 0 0.5877189 0.4546185 0.607359 1.0000000      Inf

You can also test against specific values like "int_1 > 0.5". Under the hood, hypothesis merely uses tidybayes::tidy_draws(fit\$mcmc_post) and dplyr::summarise. Read more on the mcp website and the associated pre-print. Disclaimer: I am the developer of mcp.

• +1 When I read this question four days ago I was hoping you would answer it!
– whuber
Jan 21 '20 at 17:40