2
$\begingroup$

I need some help understanding why error bars/confidence intervals (C.I.), i.e. plus/minus 1.96*standard error, are important in observation type studies. I understand them as: "If an experiment were repeated many times under the same conditions, 95% of those experiments would contain a correct parameter estimate, like a mean, and 5% of them would not. The part I find creepy is, we as the researcher, have no way of knowing whether our C.I. is in the 5% or not, especially if we don't repeat an experiment under the same conditions like with sampling in the natural environment. So, what's the point?

For something like sampling for fish, it's impossible to repeat a survey under the exact same conditions every time (time of day, location, sunlight, temp, air pressure, food source, currents, etc). Does the lack of control over factors like these (as opposed to a lab study), mean that standard deviation should be used instead (because it just describes the data we have without making assumptions about data we only collect hypothetically)?

$\endgroup$
1
  • $\begingroup$ "The part I find creepy is, we as the researcher, have no way of knowing whether our C.I. is in the 5% or not, especially if we don't repeat an experiment under the same conditions like with sampling in the natural environment." You did a good job of summarizing the major challenge/pitfall of using frequentist statistics in ecology (and the life sciences in general) using a single sentence. $\endgroup$
    – MikeyC
    Oct 29, 2021 at 20:53

1 Answer 1

4
$\begingroup$

When you report standard deviation (SD) you're describing how much your measurements vary between observations. More observations will give you a better estimate of the SD, but the estimate is not strictly dependent on sample size (see however https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation regarding the issue of bias in SD estimates).

When you report standard error, you are making a statement about how precisely you are able to estimate the population mean. The precision of this estimate is highly dependent on sample size: you get more precision in your estimates the more data you have.

Both descriptions of your data are valid, they just tell you different things. Standard deviation is a descriptive statistic, and it's always good to describe your data well. Sometimes it's better to simply display the data points you've collected, especially when there are few of them, or to report a histogram, especially when the data are not normally distributed.

Standard error is more relevant to comparisons between means, and you'll find the standard error in equations used to compare means, like the basic t-test. If you want to compare the means between different classes between observations, standard errors help demonstrate how confident you are in the group means you report.

See also https://stats.stackexchange.com/questions/32318/difference-between-standard-error-and-standard-deviation

There is no particular reason that these parameters should be used differently in observational or experimental research: both types of data can use both statistics.

$\endgroup$
9
  • $\begingroup$ What about not being able to re-sample under the same conditions? $\endgroup$
    – Nate
    Oct 29, 2021 at 16:52
  • $\begingroup$ @Nate I don't see that as particularly unique to observational studies. Experiments also have conditions that cannot be re-sampled. Describing your data using descriptive statistics is important in both. $\endgroup$ Oct 29, 2021 at 16:53
  • $\begingroup$ What's the reasoning? Just for my understanding. The standard error explicitly means that a sample was repeated many times, no? $\endgroup$
    – Nate
    Oct 29, 2021 at 16:56
  • $\begingroup$ @Nate Every data point is unique - there must be something about it that cannot be repeated; that doesn't mean those samples can't be from a broader population. If you're taking multiple samples, whether observationally or in an experimental context, you have some population you're drawing from. Both the SD and SE require you have the concept of a population in your data. $\endgroup$ Oct 29, 2021 at 16:58
  • $\begingroup$ Aha, ok, so every data point is like a "mini-sample" almost? And every sample of observations is a representation of the larger population, so a sample of sufficiently large enough observations is like taking many samples, kind of? $\endgroup$
    – Nate
    Oct 29, 2021 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.