2
$\begingroup$

What do you mean by a distribution is homoscedastic (i.e. $ σ(Y|X = x) = σ$) in the context of simple linear regression?

Why do we need this assumption in simple linear regression?

What will happen to the regression if a distribution is not homoscedastic?

$\endgroup$
  • $\begingroup$ This has been addressed in many easily accessible places. Have you done a basic search on this, and if so is there something particular that is giving you trouble? $\endgroup$ – rolando2 Mar 22 '18 at 16:23
  • 1
    $\begingroup$ yeah ,i did and i couldnt find any answers.can you give me some links to the above problem? $\endgroup$ – DRPR Mar 22 '18 at 16:29
  • 1
    $\begingroup$ I googled "linear regression homoscedasticity" and found, e.g., en.wikipedia.org/wiki/Homoscedasticity (somewhat dry), statisticssolutions.com/homoscedasticity (seems nicely written, but no graphs). Try adding the word "graphs." Good luck. $\endgroup$ – rolando2 Mar 22 '18 at 16:46
4
$\begingroup$

When you perform a regression, you are making assumptions about the distributions of the random variables whose outcome you have observed. Those observations are your data.

Homoscedasticity means that the distribution you assume is generating the $Y$ value of your data points has the same variance no matter the value of $X$.

Why do we need this assumption in simple linear regression?

The way you fit a simple linear regression model is that your look for the parameters that make the data you observed as likely as possible. This is called maximum likelihood estimation. The common recipe for finding those parameters (via algebra) works under the assumption of homoscedasticity.

Consider the following example:

Three data points are given and simple linear regression yields the following regression line:

enter image description here

Now, what if I told you that when $X$ takes the value $2$ the distribution of $Y$ has a very very small variance, same for the value $3$, while it has substantial variance given that $X$ takes the value $1$? In this case, assuming that the regression line is true, getting the data you got would be very unlikely, because the black dots are quite far from the line.

A regression line like this:

enter image description here

would give you a much greater likelihood.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ "...in generating the Y value" should say "...in generating the predicted Y value". Also, I don't view the problem with heteroskedasticity as one involving likelihood of getting these data given this regression line; rather, I see it as one of unreliability of predictions. For some values of X, Y will be much harder to predict accurately than for other values of X. Our trust in our predictions will be compromised. $\endgroup$ – rolando2 Apr 7 '18 at 19:27
  • $\begingroup$ @rolando2 why should it say "in generating the predicted Y value", when the assumption of homoscedasticity is about the underlying data generating process? $\endgroup$ – Winkelried Apr 7 '18 at 19:57
  • $\begingroup$ @rolando2 the problem the example shows is that if homoscedasticity is not given, the least-squares estimate is no longer guaranteed to be the maximum likelihood estimate. $\endgroup$ – Winkelried Apr 7 '18 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.