3
$\begingroup$

I have two sensors of different quality capturing the same process, where one of them is much more accurate than the other. Hence, I want to find out how much better.

Let us for example say that the variable from the better sensor drops by 20%, if the error interval was 5%, then the other variable is most likely to show a drop in the range 15-25%.

What I have done so far is to fit a regression line to the data points, and then calculated the distance from the values of the lower quality sensor to the regression line (and mean distance). In addition, I have also added a parallel line where 2.5% of the points are below it, and another line with 97.5% of the points below it.

Is the interval between these lines a good estimate of the error interval (prediction interval?)? Any other methods that may be more suited?

Scatterplot with regression line and error interval

$\endgroup$
  • $\begingroup$ Do you assume anything of the (non observed) value being measured. Is it known to be constant across measurements? $\endgroup$ – Benoit Sanchez Nov 17 '17 at 15:24
  • $\begingroup$ What do you know about correlation between the sensors? $\endgroup$ – David Dale Nov 17 '17 at 15:39
  • $\begingroup$ The sensors are respiratory effort belts which measure belt distraction around the abdomen or thorax, as one is breathing. The samples above are the peak belt distraction per breath. The amplitude in relation to belt distraction is assumed to be linear for the better quality sensor. $\endgroup$ – user2725580 Nov 17 '17 at 16:29
  • $\begingroup$ I think you could probably find an adaptation of Kalman filter for your problem. The exact problem statement depends on what you know (or assume) about distribution of errors and dynamics of the hidden variable. $\endgroup$ – David Dale Nov 17 '17 at 16:49
  • $\begingroup$ You mean like to use a Kalman filter to combine the output from both sensors in order to get a more optimal estimation? My goal is however to determine the quality of the lower quality sensor. $\endgroup$ – user2725580 Nov 19 '17 at 10:13
3
+50
$\begingroup$

There is some confusion in the question I'll try to clarify.

You say you want to find "how much better" one sensor is compared to the other. But the data and method only addresses finding out how far they are from each-other, which is not the same at all. The fact is the first problem (how much better) can't be estimated with these data. The second problem can be solved a bit the way you do.

To explain the key difference, I'll simplify the problem a bit and assume that each sensor is just equal to the measured variable plus noise. You measure a value $X$ with two sensors $Y_1,Y_2$ with noise. You can't see $X$. Formally:

  • $Y_1=X+\epsilon_1$
  • $Y_2=X+\epsilon_2$
  • Thus $Y_2=Y_1+(\epsilon_2-\epsilon_1)$

There is a small problem regarding the independence of noises, but ignoring this "detail", you can measure the variance of $\epsilon_2-\epsilon_1$ as the average squared difference. Adding linear dependence makes the problem a little trickier, linear regression can be used carefully. I don't focus on this.

What I wanted to clarify is that this method does not tell you how much one sensor is better than the other. This just tells you how far they are from each other. You cannot know:

  • the variance of each $\epsilon_2$ and $\epsilon_1$
  • nothing such as a ratio of these variances
  • not even which sensor is better

A case would be that $Y_1$ is perfect and the difference is $Y_2$'s fault, the other way round, or any situation in between. There is absolutely no way you can know this with such data.

The only way you can approach this problem is assuming something about the dynamics of $X$ across measurements. If you assume $X$ is constant for example, then you can estimate the variance of $\epsilon_1$ using basic variance estimator. Same for $\epsilon_2$. If $X$ evolves in a more complicated way, you can see it as an hidden state and use the Kalman filter. It's not easy to do.

$\endgroup$
1
$\begingroup$

As Benoit explains very well. The errors of the different sensors add up, making them indistinguishable, and your current measurement is missing an anchor point to find out which sensor is causing how much error.

However, you could make measurements with multiple belts (more than two). And use the method of path coefficients to find out the correlations between the unknown true respiratory effort $X$ and the measured belt distractions $Y_i$, by looking at the correlations between the different $Y_i$ (but you need more than one set of paired measurements). Your anchor point becomes the assumption that sensors of the same kind will have approximately similar variance of errors.

The expectation will be that two of the higher accuracy sensors will have a higher correlation than two of the low accuracy sensors. Once you figured out the relative size of the different errors, you can solve your original problem with Deming regression (which uses a known ratio of the variance of the error distributions).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.