# Confusion about when to use least-squares regression analysis

I am going through an article titled On the misuse of regression in earth science.

On page 65, the author say as follows about the least-squares method.

It is usual to require that the independent variable be known without error, though for purposes of prediction, the regression equation should be used if both variables are subject to error.

Then in the next section, the author says as follows for the functional analysis.

The estimation of the most probable relation underlying the data constitutes yet another problem, and it must be analyzed differently if both variables are subject to error, for it is no longer appropriate to ascribe all o f the residual variance to variation of the dependent variable.

It sounds to me that the author is suggesting both methods should be used when both variables are in error. Am I missing something?

• Isn't the author clear about the distinct objectives? "Purposes of prediction" and "estimation of most probable relation" are different. – whuber Jul 19 '16 at 21:39

The two sections you quote are not directly comparable, I think. Some background: In OLS regression, the assumptions of the process (which are not always considered by its users, as noted by the author of that article) include the requirement that the independent variable(s) be measured perfectly--that is, with zero measurement error. This condition, like many in formal statistical models, is impossible to meet in any many real data situations (not _any_... thanks, glen_b). However--like with many statistical problems--failing to meet this condition is often considered (by the experts) to have little negative consequence for data analysis. In reality, the measurement error of both the IVs and the DV are probably represented in the overall "badness of fit" of the model (the "residuals" and their many associated statistics).

Despite the above, I think the author is saying that failing to account for the reality of measurement error in all variables (i.e., IVs) can be problematic in many situations. I assume this is why there is a family of models that include estimation of measurement error in the IVs. I think structural equation modeling is like this, too, as most variables in SEM have error terms associated with them.

Finally trying to directly answer your question: The first quote is saying that in OLS regression we assume the IV is measured with no measurement error, which is clearly impossible, but (I think it's saying) that is OK when using OLS for prediction. The second quote seems to be something different: part of the description of the tasks necessary for functional analysis (probably, since that's what the article is about). In the second quote, the author is explaining the difficulty of modeling when the residual (i.e., error) variance in a model must be accurately identified as either coming from the IV or from the DV, instead of (as in OLS) just assuming it all comes from the DV. This is not directly relevant to the first quote in the context of your question.

...both methods should be used when both variables are in error.

All measured variables always have measurement error, so that statement might be based on your misunderstanding of some issues. Rather, the author seems to be saying (in the first quote) exactly what is in the article's abstract: that OLS is not appropriate in earth science except in prediction situations. The second quote is not directly related to your question, as it is a description of a part of the alternative method that the author favors.

• "impossible to meet in any real data situation" may be too strong. For example, if my predictor is categorical (e.g. treated vs untreated), in many situations it's entirely possible that it has no error. [Such situations may be very rare in some application areas of course.] – Glen_b Jul 19 '16 at 22:26
• Thanks. I failed to consider that kind of situation. You're exactly right. – DL Rogers Jul 20 '16 at 12:04

Just to clarify some points from DL Rogers' answer. OLS regression estimates the mean of the conditional distribution $E(y|x)$. Clearly, in a prediction problem, this is what we want: we are given $x$ and want to predict $y$. Since the $x$ is hypothetical, it can be assumed to be without error. In prediction, we're saying, "suppose the value of $X=x$ ... now what's my best guess for $y$?"

There are many situations in which $X$ can reasonably be assumed to have no error. Dummy variables for categories are one example. Another important class is where the $X$ values are design values from an experiment. Also, when the IV is space or time, and we know where we are and when. At a practical level, if variation in the response is several orders of magnitude greater than error in the IV's, we can safely ignore the latter.

Problems arise if the error in the IV's is comparable to the error in the response and we are interested in the correlation (or slope parameter) between the unobserved constructs. The standard regression parameter overestimates the magnitude of the true slope, so if you use OLS values when you should have used a measurement error model, you will overestimate the strength of the correlation between the constructs of interest. This would be bad.

The paper you reference is behind a paywall, so I couldn't take a look, but judging from the abstract, the author seems to be trying a measurement error model of some kind. The phrase "functional data analysis" is used by different authors to reference a variety of different methods, so I'm not sure what this paper is actually doing. For what it's worth, in the functional data analysis of Ramsay and Silverman, the IV's are assumed to be known without error.

• +1 particularly for the last point about noiseless readings. I would also add that these recordings are assumed to be "dense". Functional data analysis for sparsely and/or irregularly sampled longitudinal data is not something covered in R&S book. Also the paper (which I have not read - no access) is from 1976. I strongly suspect that the state-of-the-art has moved on since then... – usεr11852 says Reinstate Monic Aug 11 '16 at 21:34