2
$\begingroup$

I have measured some values. These values are already aggregates (in principle average values). The values have a "non-nice" distribution, for example mymeasuredvalues. They come from different sources with different setups, however they all measure the same quantity.

I now want to calculate different things.

  1. mean of the distribution - easy, done.
  2. statistical error of the overall mean. Is it just $\frac{\sigma}{\sqrt{n}}$ with $\sigma = \sqrt{Var(\text{value})}$? If so, why? I have a physics background and there we argued that natural quantities are always Gaussian-distributed. Now this here is a technical measurement. I do not see how I could apply the Gaussian errors or the central limit theorem here as we don't sum independent random variables.
  3. CLs for the mean. How would I calculate CLs that would say something like "with 95% confidence, the mean of the distribution lies within $[x-\sigma_-,x+\sigma_+]$". As you can see, I would like to have asymmetric CLs because our measurements can have upper and lower technical limits (for example, speed measurements will always be > 0) and I don't want the CLs to span the impossible value range.

And as I had some discussion with my colleagues about it (probably a new question, but still...): how can subsampling affect the calculated quantities (errors and CLs) and when should I do it? (short explanation is enough)

Thank you so much. xo.

$\endgroup$
5
$\begingroup$

Since it looks unlikely that your question will get an answer soon, I decided to have a go. This is part answer and part request for further clarification. The things I seek clarification of may give you some idea why you didn't get answers (besides the reduced presence of people around the start of January).

(Besides answering some of the question, this must go into an answer because there are far too many questions to fit into a comment, or even two.)

I now want to calculate different things.

mean of the distribution - easy, done.

Well, no, it's not done. You calculated something, sure, but it almost certainly wasn't what you just said. You can't calculate the mean of the distribution from which the data were drawn ($\mu$) unless you have the population -- and from your description it sounds like you have some kind of sample. So the thing you calculated would be a sample mean.

If that sample was a random sample* from the population of interest (and I'm not sure this is the case from your brief description) then your sample mean would be a reasonable way to estimate the population mean (the mean of the distribution).

For a random sample, the estimator $\bar{X}$ has the desired expected value ($E(\bar{X})=\mu$).

* certain kinds of non-random samples (e.g. where one identifies the subpopulations you indicate are present and then samples randomly within each of those) can be used but I'll avoid discussion of that unless it turns out to be necessary.

statistical error of the overall mean.

That depends on what "statistical error" means (it's not an especially standard term in statistics). If you mean "the standard error of the estimate of the population mean", that would be $\sigma/\sqrt{n}$, but note that you don't know $\sigma$ any more than you knew $\mu$. You can estimate it from the sample of course (typically by the sample standard deviation $s$), subject to the same requirement of random sampling.

there we argued that natural quantities are always Gaussian-distributed

That would seem highly unlikely to me (some such quantities would presumably be necessarily positive, for example). What was the basis of this argument?

Now this here is a technical measurement.

Your terminology is unfamiliar. Can you either avoid the jargon or explain its meaning? What's a technical measurement? What's a natural quantity? Why is there a distinction between them with respect to estimating a mean?

*Edit (much later): it occurs to me that you might be referring to a measurement using an instrument that has some measurement error. Is that the kind of thing you mean?*

I do not see how I could apply the Gaussian errors or the central limit theorem here as we don't sum independent random variables.

Now I am really confused. When you calculated a sample mean you didn't sum the observations along the way? Or are you saying the observations you averaged were dependent? (Or perhaps you mean to say that you didn't have very many values, in which case standardized means would not be close to normally distributed.)

measurements can have upper and lower technical limits

I'm not 100% sure what you're saying there. Do you just mean the variable being measured may be bounded (e.g bounded below, for things that are necessarily positive)?

how can subsampling

What kind of subsampling are you talking about? What is it precisely you're proposing to do?

$\endgroup$
1
$\begingroup$

First of all, the mean is not a good measure of location for everything. For uniform distributions, the average of Min$[X]$ and Max$[X]$ works better. For Cauchy distributions median works better, and properly trimmed mean better yet. For Pareto distributions with exponents $<1$, harmonic mean works better and the mean value is unphysical. I would test various measures of location of the data in an attempt to identify which of many would be more representative of location. Also, for an arbitrary f(x) provided that one such is listed, it may be possible to obtain a first moment and maybe even a second moment about the mean. But, it would still need testing for best location measure.

Second, the confidence intervals of data can be calculated directly from that data as quantiles. That is, assuming that the plot is from observed data, I would re-sample it with replacement n-fold to establish a distribution of mean values, and then take my confidence interval from that. If the curve is not from observed data, but does have an f(x), I would randomly sample f(x) to obtain n-fold means and then establish their confidence interval. I would not use that procedure for Cauchy or other fat-tailed distributions (e.g., Pareto with small magnitude exponents), for uniform distributions or other distributions for which the mean is not a good measure of location. Confidence intervals of a location parameter would be a bit more complicated, because one has to find a proper location first. Next, if all else fails, the confidence intervals of a location parameter could be estimated from bootstrap. And, symmetric they would only exceptionally be.

$\endgroup$
5
  • $\begingroup$ Could you explain, or give an example, of your assertion "confidence intervals of data can be calculated directly from that data as quantiles"? This is vague. In many cases like the one in the question it is incorrect, but there are applications (such as nonparametric CIs for population quantiles based on simple random samples) where something like what you describe is a good procedure. Could you therefore tell us what parameter these CIs are intended for, which quantiles of the data you would use, and in what circumstances your approach could be expected to work well? $\endgroup$ – whuber May 3 '18 at 13:00
  • $\begingroup$ @whuber Sure, assuming that the plot is from observed data, I would re-sample it with replacement $n$-fold to establish a distribution of mean values, and then take my confidence interval from that. If the curve is not from observed data, but does have an $f(x)$, I would randomly sample $f(x)$ to obtain $n$-fold means and then establish their confidence interval. I would not use that procedure for Cauchy or other fat-tailed distributions (e.g., Pareto with small magnitude exponents), for uniform distributions or other distributions for which the mean is not a good measure of location. $\endgroup$ – Carl May 4 '18 at 1:31
  • $\begingroup$ @Whuber Moreover, I would test various measures of location of the data in an attempt to identify which of many would be more representative of location. Also, for an arbitrary $f(x)$ provided that one such is listed, it may be possible to obtain a first moment and maybe even a second moment about the mean. But, it would still need testing for best location measure. $\endgroup$ – Carl May 4 '18 at 1:34
  • $\begingroup$ You describe a simple form of a bootstrap, but that important qualification is nowhere in your post. Apparently your phrase "that data" really means the bootstrap distribution rather than the original data. $\endgroup$ – whuber May 4 '18 at 12:57
  • 1
    $\begingroup$ @whuber Yes, that was my intent, but for politeness sake I was not about to change the text until this interlocution ran its course. Now I shall. $\endgroup$ – Carl May 4 '18 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.