In the vast majority of cases, linear regression models are used in practice as opposed to the more complicated errors-in-variables models. For the sake of example, consider modelling height $Y$ vs weight $X$, or any two appropriate continuous variable of your own choosing - the following is a typical example of what can be found in textbooks/literature: $$ Y = \beta_0 + \beta_1 X + \varepsilon. $$ As far as I am aware, this corresponds to assuming we measure $X$ with no error and we measure $Y$ with error. But we practically always have error when we measure $X$. Which in this case is weight, but in most of the cases you will find in the textbooks/literature, the independent variable comes from a measurement process so it will have some measurement error.

  1. So when we use the above model for height and weight, is the error $\varepsilon$, which explicitly accounts for measurement error in the response variable, also implicitly accounting for measurement error in $X$? Because in reality, as I just mentioned, there is usually always measurement error in the independent variable.
  2. If $\varepsilon$ is not implicitly accounting for the measurement error in $X$, then how does this lack of accounting for the measurment error in $X$ manifest itself in the results obtained from linear regression? Since this linear regression model is applied practically everywhere, it seems the deliberate mistake we are making by not accounting for measurement error in $X$ is not so bad?
  3. Finally, I have read that when the goal is prediction, errors-in-variables provides no benefit over ordinary linear regression, why is this?
  • 7
    $\begingroup$ Errors in X are ignored for (1) expediency and (2) because if you correct for such errors predictions will be off for future data that have the same degree of errors as occurred in the training data. Correction for errors in X makes regression coefficients properly farther from zero but then they apply only to future corrected X. $\endgroup$ – Frank Harrell Sep 10 '20 at 11:13
  • $\begingroup$ Suppose we could measure height $Y$ orders of magnitude more precisely than weight. In fact, suppose we could measure it perfectly. But suppose we still have measurement errors in weight $X$. Then we take measurements/samples of height and weight. How would you view the linear model $Y = \beta_0 + \beta_1 X + \varepsilon$ in this situation? Is it still generally valid to use this model? $\endgroup$ – Bertus101 Sep 10 '20 at 12:20
  • $\begingroup$ I know the correct model for my hypothetical situation in the above comment is $X = \beta_0 + \beta_1 Y + \varepsilon$, but if I want to keep height $Y$ as the response variable, can the linear model in my above comment by justified, and if so how would you interpret the error term $\varepsilon$? Note that I'm simplifying my question from something I'm dealing with in a real-life problem, hence why I want to keep height $Y$ as the response variable. $\endgroup$ – Bertus101 Sep 10 '20 at 12:21
  • $\begingroup$ @Frank Harrell: That seems like an interesting answer! Care to expand, with references to literature? $\endgroup$ – kjetil b halvorsen Sep 10 '20 at 22:54
  • $\begingroup$ Some references that seem relevant: Spurious Certainty, a systematic review, Gary King, A stored google scholar search $\endgroup$ – kjetil b halvorsen Sep 11 '20 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.