Let's say I want to know the size of some shoes, but the size isn't marked. The real size could be any of [35,36,37...50 cm], and the shoes are made so well that their real size is exactly one of these integers when measured with a reference instrument. I measure the shoes with a lower-quality instrument whose measurements include normally-distributed error. I estimated this error previously by measuring objects of known size. What is the maximum acceptable error to be able to distinguish whether a shoe is 36 cm and not 35 or 37 cm? Is it just 0.49 cm? In other words how should I specify the necessary performance of the measurement device?
$\begingroup$
$\endgroup$
13
-
1$\begingroup$ (1) How reliable do you want your distinctions to be? (2) How many times are you willing to measure a given shoe? (3) Please be aware of the difference between precision and accuracy. Your reference to the latter at the end of your question suggests there may be some systematic error in the measurement system: is that the case? $\endgroup$– whuber ♦Commented Jan 24, 2020 at 22:28
-
1$\begingroup$ @whuber (1) I would like my distinctions to be correct often, say 95% of the time. (2) I will measure a given shoe once. (3) I estimated my instrument's error previously as the RMS difference between my instrument and the master instrument on a set of objects of known sizes. Let's assume my instrument's error is randomly distributed around the real size and not a systematic bias such as always measuring 0.2 cm too high. I removed the reference to accuracy in the last line. $\endgroup$– KAECommented Jan 27, 2020 at 13:14
-
1$\begingroup$ @Ed There are several pitfalls there. The first is that the measurement error is unknown: it has been estimated from a calibration experiment. The second is that it's unlikely all shoe sizes are exact integers--that would be assuming a manufacturing process with no variability at all, which is not plausible. $\endgroup$– whuber ♦Commented Jan 27, 2020 at 15:34
-
1$\begingroup$ Assuming we can have to accept the "rounded low quality estimate" as our estimate then the maximum deviation we can accept is $0.2551$ on the sense that $1.95996*0.2551 < 0.5$. $\endgroup$– usεr11852Commented Feb 9, 2020 at 16:29
-
2$\begingroup$ @Nuclear Thank you for your comment, which is to the point. I just wish to observe that the Normality assumption is unnecessary; having a finite measurement variance suffices. And if one uses the median of the measurements, then literally any distribution with a (unique) median of zero will do. $\endgroup$– whuber ♦Commented Feb 12, 2020 at 16:53
|
Show 8 more comments
1 Answer
$\begingroup$
$\endgroup$
2
Assuming we can have to accept the "rounded low quality estimate" as our estimate then the maximum standard deviation we can accept is $0.2551$ on the sense that $(1.95996 * 0.2551)<0.5$. This would allows us to somewhat approximately ensure that the "distinctions to be correct often, say 95% of the time". Just to be clear: if our high-quality instrument is amazing accurate (say, a standard deviation of $0.001$), then having a "relative reference" to it is a bit misleading. It is more relevant to draw a performance threshold based on translating the threshold to our actual needs to make a decision and/or make direct comparison between the two standard deviations (or variances).
answered Feb 12, 2020 at 14:57
-
$\begingroup$ Can you please help me understand your last sentence? The actual need to make a decision is telling the sizes apart with the low-quality instrument, I think, so your 0.2551 answer seems to fit. Is your point that if the high-quality instrument is amazingly accurate, it's basically truth? $\endgroup$– KAECommented Mar 12, 2020 at 13:56
-
1$\begingroup$ Yes, correct; therefore using it as the baseline is not relevant in terms of a relative comparison as it has "no measurement error". $\endgroup$ Commented Mar 12, 2020 at 19:40