5
$\begingroup$

I have a number of measurements of a data point which I want to compare to my model. I have the sample mean, sample standard deviation and therefore the SEM. It usually seems fairly obvious to me whether to use the std. dev. or SEM. I think in this case I want to use the std. dev. as I'm comparing my model to my data, not estimating a mean value. However it made me wonder, if I had no model and I was simply reporting an experimental error are there any hard a fast rules as to which one would use. I have probably been rather inconsistent in the past.

I tend to prefer the std. dev, as I generally use systems which can collect thousands of discrete data points a minute, so SEM tends to be rather implausibly small compared to the uncertainty present in the measurement system. Often this is to the point where the screen/printer resolution isn't sufficient to distinguish the upper and lower bounds using the SEM. If I work on an experiment where repeated measurements are hard to come by my mind seems to go to SEM. Is it simply a case of honesty? If stats are your dominating uncertainty quote SEM and if its measurment error quote std. dev.?

I feel like I should not be running on auto-pilot on what seems like an important question.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
9
$\begingroup$

Absolutely it is not just a question of honesty, or anything to do with stats v measurement error. The standard error of the mean and the standard deviation of the population are two different things.

The mean of your sample is a random variable, because it would be different every time you ran the sampling process. The sampling error of the mean is just the estimated standard deviation of the sample mean.

It's not quite clear what you mean by comparing data points to your model. But if you mean you are interested in whether a particular data point is plausibly from the population you have modelled (eg to ask "is this number a really big outlier?), you need to compare it to your estimate of the population mean and your estimate of the population standard deviation (not the sample mean's standard deviation, also known as SEM). So the standard deviation in this case.

More generally, it sounds like you are using the standard deviation inappropriately in some other circumstances. If you are trying to report inferences about the population mean you should use the sample mean's standard deviation / standard error, not the population standard deviation.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I understand the conceptual difference but in practice the kind of situation where I typically get confused is where I have a model of a system which predicts a value. To validate/refute my model I make repeated measurements at a given point. I can take data sufficiently quickly that within a few minutes the SEM is an implausibly small number that doesn't seem to tell me anything other than that dividing a small number by a big one gives a very small number. I guess its just telling me that I can't control the parameters of the experiment as well as I can take data. $\endgroup$
    – Bowler
    Commented Jan 17, 2012 at 9:30
  • $\begingroup$ Which brings me back to which should I use to determine how well my model matches my data. I would like to think of my model as an estimator of the mean, I'm not sure how else I would interpret it. When I compare the sample standard deviation to the predicted model the agreement is well within one standard deviation, but by virtue of the number of data points the SEM refutes the model by quite a few std. devs of the mean. Although as this is so small this number is rather eratic. Maybe I asked the wrong question maybe my question should have been, what is this situation telling me? $\endgroup$
    – Bowler
    Commented Jan 17, 2012 at 9:43
  • $\begingroup$ It sounds like the mean of your new data is much more than 1.96 SEMs than the predicted value of your model. In this case, you have successfully refuted your model. If your problem is that your massive data gives you precision beyond what is material (eg sure 50.0000001 is different from 50, but does it matter?), you might want to try a Bayesian approach which allows your parameters a distribution rather than the implausible point value insisted on by frequentist approaches. $\endgroup$
    – Peter Ellis
    Commented Jan 17, 2012 at 18:23
  • $\begingroup$ I suggest you ask a new question along the lines of your second comment. Using the SD instead of the SEM is simply wrong, so your question is more to do with the appropriateness of your model and whether the mean is really a constant. On reflection, I suspect what is happening is that there is no single value of the mean but it changes from time to time. When you take what look like thousands of sample points, they are not independent samples from the overall distribution, but just from where the distribution is at that particular point. $\endgroup$
    – Peter Ellis
    Commented Jan 17, 2012 at 18:26
  • $\begingroup$ This could be addressed either by having a model that allows the mean to change from time to time, or some other method of allowing for the dependence of the points in your sample. $\endgroup$
    – Peter Ellis
    Commented Jan 17, 2012 at 18:28
-1
$\begingroup$

I think of it this way; the SEM is a measure of the precision of a sample mean. But a sample mean from one experiment (say, with 3 -6 replicates) is hardly enough to gauge the precision of a population mean. FOr me, one needs to do multiple independent experiments (each with replicates to generate a mean) and then each of these means are taken as individual n's. So, instead of individual samples (replicates from an experiment), I only use the individual means of several (at least 3, more often 4-6) independent experiments to calculate the SEM and I use the STD of those averaged means in the calculation of an SEM. SInce few (including me) perform experiments more than 3 times, I elect to use the STD of a "representative" experiment. I think the SEM is not very useful and most people use it simply to reduce the size of the error bar. Therefore, I do not like the SEM much and loathe its pervasive use in science.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The population mean doesn't have a precision; it's just a number. $\endgroup$
    – Hong Ooi
    Commented Jul 22, 2015 at 5:50
  • $\begingroup$ People use the standard error to use the sample to make inference about the population mean. The standard deviation is used to describe how variable your sample is. Those are two very different aims. You don't need multiple experiments to compute the standard error. Though there are other good reasons for replicating experiments, computing standard errors is definately not one of these reasons. $\endgroup$
    – Maarten Buis
    Commented Jul 24, 2015 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.