6
$\begingroup$

I have a set of $N$ points $(x_i,y_i)$. $X$ and $Y$ both have some noise associated with them due to measurement inaccuracy however the relationship of the underlying true values (i.e. if we could remove the noises) of these points should be of the form $y = mx +c$ where $m$ and $c$ are constants.

However, due to measurement inaccuracies in both $X$ and $Y$, I will get uncertainties in both my $m$ and $c$ values.

1)If I assume that my measurement inaccuracies for both $X$ and $Y$ are Gaussian distributed $\epsilon$ ~ $N(0,\sigma)$ how do I obtain the most likely $(m,c)$ and the uncertainties/confidence in both.

2) If I instead know that the uncertainties are different for $X$ and $Y$ such that $\sigma_x \neq \sigma_y$ where $\epsilon_x$ ~ $N(0,\sigma_x)$, $\epsilon_y$ ~ $N(0,\sigma_y)$ can I get a different estimate of $(m,c)$ and the uncertainties/confidence in both.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Search our site for errors in variables regression and model II regression $\endgroup$
    – Glen_b
    Aug 18, 2019 at 2:00
  • $\begingroup$ Unless you have perfect correlation, you will have uncertainties in m and c even without uncertainties in X and Y. $\endgroup$
    – Matti Wens
    Aug 7, 2020 at 12:58

2 Answers 2

Reset to default
4
$\begingroup$

In both cases you want to use Deming regression. Case 1 is a special case of Deming regression called orthogonal regression, which minimizes the sum of squared perpendicular distances from the data points to the regression line. For case 2, the general case, you will need an estimate of the ratio $\delta = \sigma^2_y / \sigma^2_x$ for the problem to be solvable.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Interesting. Thanks for your response - I will look at the links and see if they answer my question. $\endgroup$
    – piccolo
    Aug 17, 2019 at 8:41
  • $\begingroup$ Thanks. The wiki page doesn't mention anything about calculating the uncertainties in the estimates of $m$ and $c$. $\endgroup$
    – piccolo
    Aug 17, 2019 at 9:51
  • 4
    $\begingroup$ I've heard this recommendation before, but this paper (Smith 2009) argues that error in X is a bad criterion for choosing RMA regression (which I think is the same as Deming regression). It argues instead that symmetry is a better criterion for whether to choose OLS or RMA, and proposes a few alternatives for dealing with the problem of error in X. $\endgroup$
    – mkt
    Aug 17, 2019 at 9:57
7
$\begingroup$

As a general concept the problem of error in X is called measurement error.

In linear regression analysis it causes attenuation bias, which is considered as one of the sources of engogeneity. Measuremet error shrinks coefficient of the right-hand-side variable measured with an error towards zero. It causes not uncertainty of an estimator, but its inconsistency instead.

While mentioned in other answer deming regression is two-variable concept, the multivariate solutions include instrumental variable method as preferred option.

Formulas for attenuation bias in case of linear regression are precisely derived, for example in here. This means if you have a clue of the variation of the error, you might estimate severity of the problem and possibly correct for it.

Measurement in Y, in case of linear regression is less of the problem, as linear regression assumes random error in the dependent variable. It causes worse prediction and higher residual variance, but do not bias coefficients in any way.

EDIT: This problem is directly described in Hausman (2001).

Hausman, Jerry. "Mismeasured variables in econometric analysis: problems from the right and problems from the left." Journal of Economic perspectives 15, no. 4 (2001): 57-67.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks. How would you get an uncertainty in the estimates of $m$ and $c$ when you use Deming regression? $\endgroup$
    – piccolo
    Aug 17, 2019 at 9:50
  • 1
    $\begingroup$ As for deming regression, here are some nice notes which include variance formulas - if a variance or sd is a measure of uncertainty of what you are looking for: ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/… If looking for technical solution, I would suggest R: r-bloggers.com/deming-and-passing-bablok-regression-in-r $\endgroup$
    – cure
    Aug 17, 2019 at 10:03
  • $\begingroup$ The last link suggest also Passing Bablok method. I did not mention it in my answer, but I guess it would be an interesting area of investigation here. $\endgroup$
    – cure
    Aug 17, 2019 at 10:09
  • 2
    $\begingroup$ A word of warning about Deming regression: confidence intervals for the slope parameter are not guaranteed to be bounded, i.e. they can include $\pm{\infty}$. This can occur when the signal-to-noise ratio in the data is too low. $\endgroup$
    – Estacionario
    Aug 17, 2019 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.