# How do I compare the accuracy of two measurement devices when one of them is the reference?

I'm currently trying to compare the measurement accuracy of two devices. However, I am using one of the devices as a "gold standard" to say something about the accuracy of the other device.

The quantity I am trying to measure is the distance (depth) to a point in a scene (image). I am using a laser scanner which gives me (x,y,z) coordinates for a point and I am using optical geometry to give me (x,y,z) coordinates for the same point. My base (or reference) measurement is the laser scanner measurement. I am using this as a gold standard. I now have a corresponding measurement from the optical geometry based measurement system. I have corresponding measurements for 1000's of points. I average the error in the difference between these and get a mean error. This error, however, neglects the fact that the laser scanner too is a measurement system and has its own error. It comes with an accuracy of +/- 5mm for an object at 10m range. How do I incorporate this into an accuracy metric for the optical geometry based system?

-

 I see, so you are saying Var(X-Y) = $\frac{\sum_{i=1}^{N} (X_i - Y_i)^2}{N}$ ? And that Var(X) = Var(X-Y) - Var(Y)? – Mustafa Jul 3 '12 at 20:44 Yes because I am assuming the random components of the two measurement system estimates are statistically independent and hence Cov(X,Y)=0. – Michael Chernick Jul 3 '12 at 20:48 OK, I get that, thanks. Maybe a related question I should be asking is that what is a good metric to use to comment on the accuracy of a measurement? Let's say I am measuring length. Let Y be the truth and X be the measurement. Var(X-Y) tells me the variability of my measurement about the mean. Does that mean that the E[X-Y] (expected value) should be close to zero. Intuitively, if it is non-zero, then it means that there is a bias to the measurement right? E[X] = E[X-Y] + E[Y] right? – Mustafa Jul 3 '12 at 20:56 @Mustafa That would be yes. – Michael Chernick Jul 3 '12 at 21:03