2
$\begingroup$

I have a physics experiment where I measured the velocities of an object. There were many repetitions to the experiment, with the assumption that the expected value of the variable is the mean of all the measurements.

As part of the data analysis, I plotted a histogram of $v_i - \bar{v}$, i.e. the deviation of every data point from the expected value. The histogram turned out as such:

enter image description here

My question is, is it true that I can only claim my variable has a random error, rather than systematic, if and only if the error distribution above is a normal distribution? My guess is that this is the case, and the closer the above is to a normal distribution, the smaller the systematic error and impact of anomalous measurements in the dataset and the better it will be at making a true prediction.

Moreover, is the frequency spike in the $[0.02,0.04]$ range a sign that there was some sort of predictable bias to the results? If so, are there methods of correcting this to get a better expected value?

$\endgroup$

2 Answers 2

2
$\begingroup$

As a general principle it is not necessarily true that random measurement errors are mean zero or Normal or constant variance. Some examples

  • scintillation counters and Geiger counters have error coming from a Poisson process, which is not symmetric
  • the true value of the lung function measurement FEV$_1$ is defined as the maximum achievable by the individual, so measurement error is always negative
  • for intrinsically non-negative quantities with large range, errors may be closer to Normal and constant variance on a log scale

However, people put a lot of effort into measurement protocols and devices, especially in the physical sciences, that do give approximately mean-zero, constant-variance errors. And some protocols involve taking differences between two measurements, either before and after sample collection or exposed vs control. These should give error symmetric around zero if done well. When you see something isn't symmetric and mean zero, it's worth asking why.

You don't say how the measurements were made in your data, but I do think the spike for small positive values suggests a problem. Without more information I wouldn't want to comment on whether you should expect the residuals to have heavier or lighter tails than a Normal distribution.

$\endgroup$
0
$\begingroup$

I agree with @ThomasLumley (+1) that it is worthwhile asking why your data are not normally distributed. In particular, you might ponder why there is a spike just above the mean. Perhaps a literature search on similar measurements will explain what is going on and even suggest what kind of distribution you should anticipate.

I have (roughly) reconstructed data that might be similar to the data you had before you subtracted the sample mean, speculating that the mean is around 20. (Of course the value 20 is entirely fanciful, but I needed to use something in order to provide a useful discussion how to deal with your actual data.) I have the data in my R session as a vector v of $n = 202$ measurements. Summary statistics and a frequency histogram are as follows:

summary(v)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  18.90   19.70   19.91   19.96   20.30   20.70 
length(v); sd(v)
[1] 202
[1] 0.385954
hist(v, ylim=c(0,50), label=T)

enter image description here

Partly because of the 'spike', it is no surprise that these data fail a Shapiro-Wilk normality test with a P-value far below 5%.

shapiro.test(d)

        Shapiro-Wilk normality test

data:  d
W = 0.94772, p-value = 1.025e-06

A density histogram of the data with the 'best fitting' normal curve is shown below. [With sample size as small as $n=200,$ you should not expect a histogram, even of normal data, to show a perfect fit to a normal density curve, but your data are clearly far from normal.]

hist(v, prob=T, col="skyblue2")
  curve(dnorm(x, mean(v), sd(v)), add=T, col="red")

enter image description here

If data were nearly normal, then a 95% confidence interval (CI) for the mean $\mu_v$ of the population from which these data were sampled would be of the style $\bar V \pm t^*S_v/\sqrt{n},$ where $t^*$ cuts probability $0.025 = 2.5\%$ from the upper tail of Student's t distribution with 201 degrees of freedom, and $S_v$ is the sample standard deviation. (Here $t^* \approx 2.0$ and $S_v = 0.386$ is given in the data summary above.) In R, the 95% CI computes to $(19.91, 20.01).$ However, data are so far from normal that one wonders whether this CI is valid.

t.test(v)$conf.int
[1] 19.90788 20.01497
attr(,"conf.level")
[1] 0.95

A 95% nonparametric CI uses only the data and the assumption that the population has a mean. By taking repeated 're-samples' of size $n = 202$ with replacement from the data vector v the bootstrap procedure assesses the variability of the data about the sample mean to give a useful interval estimate of the population mean. The resulting 95% nonparametric bootstrap CI for $\mu_v$ is $(19.91, 20.02).$ It happens to agree closely with the CI above (which assumes normality).

set.seed(902)
a.obs = mean(v)
d.re = replicate(3000, mean(sample(v,202,rep=T)) - a.obs)
a.obs - quantile(d.re, c(.975,.025))
   97.5%     2.5% 
19.90853 20.01564

Note: I simulated fake data that I hope is not too different from your actual data as follows. (For future reference, when asking statistical questions, it is best to provide your actual data in addition to your summaries or histogram---unless you have good reasons not to reveal this information.)

set.seed(2020)
dif = seq(-11,7, by=2)/10; f = c(1,1,13,30,29,29,27,47,22,3)
di = rep(dif, f);  u = round(runif(202, -.01, .01),3);  d = di + u
v = 20 + d
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.