6
$\begingroup$

Let's say I measured the weights of 50 chickens from my family farm, which keeps 1000 chickens. The sample mean is 5 kg, SEM is ± 3 kg, and the 95% confidence interval is 5 ± 3 * 1.96 = -0.88 kg to 10.88 kg. How should I interpret the results of SEM and CI. Obviously the weight of a chicken should not be negative.

  1. It seems to me SEM has little use except to calculate CI? What quantitative information can we derive from SEM? We can say the true mean weight of the 1000 chickens is likely (very qualitative) to fall between 2 kg to 8 kg (sample mean ± SEM), but do we know the probability?
  2. How to interpret the negative lower bound of CI?
  3. How much probability that the true mean weight will fall in the range between 0 kg - 10.88 kg?
$\endgroup$
1
  • 3
    $\begingroup$ That seems implausibly high for SEM.... Some monster chickens you have there. $\endgroup$
    – Nick Cox
    Commented Oct 3, 2019 at 7:23

4 Answers 4

2
$\begingroup$

It seems to me SEM has little use except to calculate CI? What quantitative information can we derive from SEM? We can say the true mean weight of the 1000 chickens is likely (very qualitative) to fall between 2 kg to 8 kg (sample mean ± SEM), but do we know the probability?

Let us begin with an observation. The SEM is not a descriptive statistic. It is derived from the data. It informs you about the sampling error of the statistic but not the uncertainty in the population. It is an artifact of the measurement process.

Had you chosen a different measurement, such as the median, you would have had different standard errors. Likewise, had your model been different, you would have had different standard errors.

There is an infinite number of possible confidence interval functions. You are using the standard one from a textbook, but it is not the only one. It is a model that has desirable properties, so it is taught, but there could be a different interval if you chose to formally model losses you would obtain from getting a bad sample.

The SEM is providing sample-specific information. For the purposes of your question, its only use is as an interim step in a calculation.

Confidence intervals tell you the area you have confidence in for the location of the mean (or some other statistic). Confidence intervals tell you nothing about the distribution of the sizes of the chickens themselves.

The interval you may want is the tolerance interval. If you wanted to know the range where 95% of your population of chickens is likely to fall, then you want the 95% tolerance interval and not the 95% confidence interval.

How to interpret the negative lower bound of CI?

The bounds of a confidence interval have no interpretation. They are random numbers. A function that generates an interval is an $\alpha$ percent confidence interval if, upon infinite repetition, the interval would cover the true value of the parameter at least $\alpha$ percent of the time.

If you create an $\alpha$ percent confidence interval and it is $[a,b]$ then the interpretation is that if you behave as if the true value were inside that range then you would be made a fool of less than $\alpha$ percent of the time once repetitions became very large.

A negative bound is fine. Let's imagine that we are Mother Nature, and you know the true population mean is at 4 kg. You should be delighted then because the interval $[-.88,10.88]$ contains the actual value. The lower bound is indeed non-sense, but Frequentist methods allow non-sense answers as long as it covers the true value a certain percentage of the time.

Also, note that narrow intervals are not better than wider ones. Narrow ones are not more accurate than wide ones. They are equally precise in that they cover the true value at least a fixed percentage of the time on large repetition.

To see why, imagine that you divided the population of chickens in half randomly and weighed them. One-half of the chickens had a narrower interval than the other half. What about the randomization process made one group more accurate? Nothing.

How much probability that the true mean weight will fall in the range between 0 kg - 10.88 kg?

That is a model-specific question. I would be concerned that your data is not normally distributed. While they are probably normally distributed, given roughly equal ages and diets, the population contains chicks and very old chickens. I would be surprised to find that they were normally distributed on an uncontrolled basis.

However, if we pretend that the chickens are sufficiently similar to each other to be normally distributed, then we can start to address your question.

First, a confidence interval is not a statement of probability. If you want a probability, then you will need to use a Bayesian model. A Bayesian credible interval will tell you the probability that a parameter is inside some range. Frequentist methods will not do that.

The reason is that there is either a 100% or a 0% chance that the parameter is inside the range, in Frequentist thinking. In Frequentist thinking, you cannot make a probability statement about a fact.

George Washington either was the first President, or he was not. That is a factual question and not subject to probability statements. A Frequentist cannot say, "it is probably raining." A Bayesian can. It is either raining, or it is not. The parameter is either inside the range, or it is not.

What you can say is that you have 95% confidence that the interval covers the parameter. What you cannot say is that there is a 95% chance that the parameter is inside the interval. That is not true.

What you have confidence in is the procedure and not the data. Your data is a random collection. There is supposed to be nothing special about it. As such, your interval and sample mean are random too. There is nothing special about them either. The population parameter, $\mu$, is special. What makes a sample mean or a confidence interval special in any sense is their relationship to $\mu$.

They summarize the information you have gathered about $\mu$ but are not $\mu$. The procedure gives you guarantees, if your model is valid, about how often you will make incorrect decisions and take incorrect actions based on the sample that you saw.

Even tolerance intervals require you to state how often you want to be made a fool of. There is no absolute tolerance interval; there are only intervals given $\alpha$, the data, and the model.

$\endgroup$
7
  • $\begingroup$ Thank you so much for your detailed answer, really helped me a ton, I'd give you a hug if I can! $\endgroup$
    – Elizabeth
    Commented Oct 4, 2019 at 6:53
  • $\begingroup$ A lengthy and vigorous statement doesn't save one bad argument here (among several valid and thoughtful points). You express unconcern about a negative confidence limit. Without going full Bayes, it is clear that prior knowledge excludes negative weights as absurd. Dismissing that implication as irrelevant or unimportant just looks silly to many non-statisticians. I am one and I wouldn't even try to defend it to my colleagues or students. At best it looks like a mathematical joke as when -2 people pops out of a puzzle as an answer that fits the information. $\endgroup$
    – Nick Cox
    Commented Oct 4, 2019 at 7:48
  • $\begingroup$ (ctd) I suggest that the aim of any interval is to be candid quantitatively about uncertainty and to do that consistently with the data and with the precise goal of analysis, but also ideally to do that in a principled and easy to explain way. Manifestly, that leaves room for different solutions, including discussion about all those criteria. I regard 95% (or whatever other fraction) in a frequentist CI, or any other interval, as uncertainty that I am painting where it belongs in a picture of the data. Putting paint in silly places helps no-one. $\endgroup$
    – Nick Cox
    Commented Oct 4, 2019 at 7:50
  • $\begingroup$ @NickCox actually knowing absurd results are possible is quite important to understanding the methodology. It is a warning. You are correct, there is a Bayesian improvement easily possible here. Nonetheless, there can be reasons not to use a Bayesian method such as a sharp null hypothesis. $\endgroup$ Commented Oct 4, 2019 at 13:34
  • $\begingroup$ @NickCox you want paint in silly places because it is a warning that something may be amiss. There are other ways to construct CIs based on other loss functions. Still, the CI she chose does what she intended it to do. It covered the parameter, maybe. A non-parametric interval would have been definitively positive because there were no negative observations. There are other possibilities as well. $\endgroup$ Commented Oct 4, 2019 at 13:36
4
$\begingroup$

what you did - you created confidence intervals under assumption that chicken weights are drawn from normal disrtibution (with value range $(-\infty, \infty)$) - in fact these can be drawn from other disrtibution with $\mathbb{R_+}$ support e.g. erlang or chi distribution, but when sample size is $> 50$ we can assume that mean has normal disrtibution - so this value $-0.88$ is effect of that assumption, so you can interpret it as 0... but to do it in strict mathematical way you should find real distribution for chicken weights, then construct propper confidence intervals (which will be different than for normal distribution) and then you will have more accurate estimates and you will drawn= more meaningfull conclusions,

but remember that conclusions you will draw will be conclusions about that sample of 1000 observations you already have!

$\endgroup$
4
  • 2
    $\begingroup$ (+1) to elaborate on this, your 95% CI tells you that in repeated samplings of your chickens, about 95% of the time will the sample mean fall between -0.88 and 10.88 kg. However, you also know that it will NEVER be less than 0, because the weights are always positive. Thus it also holds that [0,10.88] is a 95% CI. quester is right that you might be able to get better inference with an explicit model, but just using this trick is probably okay for an informal application like this. $\endgroup$ Commented Oct 3, 2019 at 6:20
  • $\begingroup$ @SheridanGrant one thing this CI is about history observations... it's very important to think about statistics as a way to describe history not the future..., although extrapolation is only at your own risk since environment of future observations can change $\endgroup$
    – quester
    Commented Oct 3, 2019 at 6:24
  • 1
    $\begingroup$ there's no extrapolation here, the definition of a frequentist confidence interval is in terms of repeated samplings from the same process that produced your sample. This is fully mathematically precise. $\endgroup$ Commented Oct 3, 2019 at 6:28
  • 1
    $\begingroup$ Sample size of 50 is pretty arbitrary, and is not really valid with high confidence (e.g., for 99.9% CI), or small alpha (e.g., $\alpha$ = 0.001). $\endgroup$
    – Alexis
    Commented Oct 3, 2019 at 15:34
3
$\begingroup$

I would just (first of all) work on logarithmic scale and back-transform the confidence limits obtained on that scale. That way you're assured of positive limits.

Going full Bayes on this is an answer of wide appeal, but as you're asking this question I am not clear that "learn a whole new approach to statistics" is likely to be practical immediately for you.

All confidence limits are at best smart guesses. But it's clear that a negative lower limit is biologically absurd, so you owe it to science to avoid that if possible. I don't go with those who say "just round up to zero". The technique is inappropriate if it produces absurd results.

More generally, a scale on which the data are symmetrically distributed will produce more sensible results than those you cite. A square root or a cube root scale might work better than a logarithmic scale in some cases.

Some of this advice depends on taking your example fairly literally. What's axiomatic is that using logarithms first is guaranteed to yield positive upper and lower limits.

(I regard this answer as consistent with advice to consider a generalized linear model with appropriate family and non-identity link.)

PS Why not bootstrap a CI?

$\endgroup$
11
  • $\begingroup$ Hi, thank you for your reply, I am so happy to see an answer not suggesting Bayes. Can you be more specific about the log transformation? What data should be transformed? like exp ( ln( 5) ± ln(3) * 1.96 ))? is there any literature supporting this logarithms-exponent transformation? $\endgroup$
    – Elizabeth
    Commented Oct 3, 2019 at 9:22
  • 1
    $\begingroup$ Transform all the data. Get confidence intervals on that scale. Transform back. Do not follow your recipe, which is not correct. If you are using decent software, it's one line. You don't have to calculate SE separately. $\endgroup$
    – Nick Cox
    Commented Oct 3, 2019 at 10:30
  • $\begingroup$ "one line" could be an exaggeration, but given that you have data in a file, taking logarithms, applying a confidence interval calculation and back transforming the limits are the steps. But note what you're doing is getting a CI for the geometric mean. $\endgroup$
    – Nick Cox
    Commented Oct 3, 2019 at 10:58
  • $\begingroup$ In general inference (e.g., CIs) on transformed data (i.e. $f(x)$) ≠ inference on untransformed data (i.e. $x$) because $\sigma^{2}_{f(x)} \ne f(\sigma^{2}_{x})$ in general (and same is therefore true for SEM), so you can't simply "back-trasform" confidence limits to get valid estimates on untransformed data. Bootstrap is a good approach. $\endgroup$
    – Alexis
    Commented Oct 3, 2019 at 15:29
  • $\begingroup$ @Alexis Indeed; the procedures are not equal. My leading suggestion -- which is tentative given the question so far -- boils down to an assertion that a CI for the geometric mean may make more sense for a variable necessarily positive and evidently skewed. If you're dismissing estimating on a transformed scale followed by back-transformation, you are dismissing a vast amount of good practice. Note: I am suggesting transforming all the data, not the mean or SE, as explained in my answer. I know that e.g. the mean or SE of logarithms is not in general the logarithm of the mean or the SE. $\endgroup$
    – Nick Cox
    Commented Oct 3, 2019 at 16:13
1
$\begingroup$

To produce a probability for weights you should really apply Bayesian methods in this case. This is not about frequentism against Bayes, but you have some very strong prior information here: You know, that a chickens weight is not negative and that it is not .5 kg. Standard frequentist methods are basically open for all results, often presuming normally distributed data and your example is a good example for non-normally distributed data.

Find yourself a credible prior distribution that excludes negative chicken (how about half-normal prior?) and compute a posterior distribution. From that posterior distribution you can conclude real probabilities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.