3
$\begingroup$

I'm studying the review for one of my graduate classes, Correlation and Regression Analysis, and this is one of the questions:

A friend says that you don't have to worry about the assumption that the residuals are uncorrelated with each predictor variable: each time she has conducted a regression analysis, the residuals always turn out to be uncorrelated with the predictors by mathematical necessity. Explain for your friend (a) what is really meant by this assumption, (b) why your friend came to the wrong conclusion trying to detect violations of the assumption in the manner described, and (b) the consequences of violating the assumption.

I'd really appreciate some help with it.

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Welcome to Cross Validated! Please add the self-study tag, read its tag-wiki and modify your question to follow the guidelines on asking such questions. In particular, you'll need to clearly identify what you've done to solve the problem yourself, and indicate the specific help you need at the point you struck difficulty. $\endgroup$
    – Glen_b
    Commented Oct 26, 2015 at 3:48
  • $\begingroup$ Can you provide the link to this paper or book? $\endgroup$
    – Xi'an
    Commented Oct 26, 2015 at 9:31

1 Answer 1

Reset to default
6
$\begingroup$

I think the author of this question is confused about the distinction between errors (unobservable quantities that in the model shift observed values from their conditional expectation) and residuals (sample quantities that are calculable, and which are distinct from the errors, though related to them).

"Residuals are uncorrelated with the predictors" is not an assumption for regression. If they've told you that it is, they're wrong, and there's no real way to handwave this error away. It's wrong at a pretty fundamental level.

There is an assumption about the errors (which are not the residuals) being uncorrelated with the predictors.

As the hypothetical friend in the problem suggests, the residuals are uncorrelated with the predictors by construction and there's no issue there. The friend's conclusion is correct and the premise of the question is false.

There's no difficulty that is not caused by the author of the question apparently conflating errors with residuals.

I'd be very interested to see what the materials you have no doubt already been given on this subject have to say on this issue.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ The question as written is arguably a trick question, but for all we know, that's the point -- not that that would make it a good question (it wouldn't), but my point is that charging the author with being fundamentally confused and speculating about what department they might be from seems a little unwarranted. I think worst case scenario we're looking at a sloppy use of terminology here. $\endgroup$
    – Jake Westfall
    Commented Oct 26, 2015 at 6:17
  • 1
    $\begingroup$ @Jake I really don't think we can write it off as simply sloppy terminology, because they make such a specific song and dance about it -- their answer to "but residuals are mathematically uncorrelated with predictors" wasn't to say "Oh, yeah, we're actually not talking about residuals", but to write a question where they accuse the hypothetical student (no doubt standing in for real people) of failing to understand their point, which they still maintain is about residuals to the extent that they're making students produce an argument relating to it in revision on it. ... ctd $\endgroup$
    – Glen_b
    Commented Oct 26, 2015 at 8:30
  • 1
    $\begingroup$ ctd ... so it seems they're going out of their way to insist they really do mean residuals - and they're definitely wrong about that, and I can't excuse it. I'm all for giving a bit of leeway on an honest mistake, but it looks like people have attempted to point out the problem to them before, and instead of responding to it correctly, they've doubled down on their original usage. They have insisted it's residuals they mean in the face of the correct response to that usage, and not only do they persist, but they insist their students learn it and repeat it. There's really no excuse for that. $\endgroup$
    – Glen_b
    Commented Oct 26, 2015 at 8:33
  • 1
    $\begingroup$ I'll allow that there's a remote possibility that it might be a trick question (if it is, my answer would work as a response), but I'll bet five dollars it really isn't; I take them at their word, and I bet that's what they intend. (If that was a trick question it's even more egregious.) I'd like to be wrong on this, but I'll put my money where my mouth is. However, I've removed the comment under my answer about particular kinds of application-areas tending to make particular kinds of errors and pass them down. (I'd also like to be wrong about that.) $\endgroup$
    – Glen_b
    Commented Oct 26, 2015 at 8:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.