8
$\begingroup$

In the vast majority of cases, linear regression models are used in practice as opposed to the more complicated errors-in-variables models. For the sake of example, consider modelling height $Y$ vs weight $X$, or any two appropriate continuous variable of your own choosing - the following is a typical example of what can be found in textbooks/literature: $$ Y = \beta_0 + \beta_1 X + \varepsilon. $$ As far as I am aware, this corresponds to assuming we measure $X$ with no error and we measure $Y$ with error. But we practically always have error when we measure $X$. Which in this case is weight, but in most of the cases you will find in the textbooks/literature, the independent variable comes from a measurement process so it will have some measurement error.

  1. So when we use the above model for height and weight, is the error $\varepsilon$, which explicitly accounts for measurement error in the response variable, also implicitly accounting for measurement error in $X$? Because in reality, as I just mentioned, there is usually always measurement error in the independent variable.
  2. If $\varepsilon$ is not implicitly accounting for the measurement error in $X$, then how does this lack of accounting for the measurment error in $X$ manifest itself in the results obtained from linear regression? Since this linear regression model is applied practically everywhere, it seems the deliberate mistake we are making by not accounting for measurement error in $X$ is not so bad?
  3. Finally, I have read that when the goal is prediction, errors-in-variables provides no benefit over ordinary linear regression, why is this?
$\endgroup$
5
  • 9
    $\begingroup$ Errors in X are ignored for (1) expediency and (2) because if you correct for such errors predictions will be off for future data that have the same degree of errors as occurred in the training data. Correction for errors in X makes regression coefficients properly farther from zero but then they apply only to future corrected X. $\endgroup$ Sep 10, 2020 at 11:13
  • $\begingroup$ Suppose we could measure height $Y$ orders of magnitude more precisely than weight. In fact, suppose we could measure it perfectly. But suppose we still have measurement errors in weight $X$. Then we take measurements/samples of height and weight. How would you view the linear model $Y = \beta_0 + \beta_1 X + \varepsilon$ in this situation? Is it still generally valid to use this model? $\endgroup$
    – Bertus101
    Sep 10, 2020 at 12:20
  • $\begingroup$ I know the correct model for my hypothetical situation in the above comment is $X = \beta_0 + \beta_1 Y + \varepsilon$, but if I want to keep height $Y$ as the response variable, can the linear model in my above comment by justified, and if so how would you interpret the error term $\varepsilon$? Note that I'm simplifying my question from something I'm dealing with in a real-life problem, hence why I want to keep height $Y$ as the response variable. $\endgroup$
    – Bertus101
    Sep 10, 2020 at 12:21
  • $\begingroup$ Some references that seem relevant: Spurious Certainty, a systematic review, Gary King, A stored google scholar search $\endgroup$ Sep 11, 2020 at 14:08
  • $\begingroup$ Related question on why you should worry about measurement errors. $\endgroup$
    – Durden
    Jul 9 at 2:48

3 Answers 3

2
$\begingroup$

Errors in X are ignored for (1) expediency and (2) because if you correct for such errors predictions will be off for future data that have the same degree of errors as occurred in the training data. Correction for errors in X makes regression coefficients properly farther from zero but then they apply only to future corrected X. I wish I had a reference for this. A simulation may be in order.

$\endgroup$
0
$\begingroup$

To add to Frank's point of expediency: a model with measurement error in the predictor is unidentified (based on a result by Solari) without additional assumptions. You either need to know the variance of the measurement error (which you generally don't), or you need repeated measurements of each $X$ such that the error variance can be deduced (which you generally don't have).

$\endgroup$
0
$\begingroup$

In short: it helps to measure both response and predictor variables precisely.

This Wikipedia article on regression dilution should help. Measurement error in X 'flattens' the ordinary least squares line (increases the intercept and reduces the slope in your example):

Is correction necessary? In statistical inference based on regression coefficients, yes; in predictive modelling applications, correction is neither necessary nor appropriate.

I can add that regression dilution can't occur if the intercept is fixed (e.g. regression through the origin) although I don't have a reference for that.

You may then read the Wikipedia article on errors in variables models. However, as noted in Durden's answer, we don't always have repeated measurements of X.

When we do have repeated measurements of predictor variables, it might be simpler to average those (being more precise: SE(mean) = SD(mean)/sqrt(n)) and then use OLS (or other common modelling approaches that ignore measurement error in X) rather than use an errors in variables model (which can be unfamiliar to readers). It depends on what the goal of the modelling is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.