2
$\begingroup$

I'm doing a simple linear regression project (only one feature), and I am checking the assumptions, however, I don't really understand the reasoning behind our homoscedasticity assumption. I've looked all over for a solution, but all of the answers are hard for me to understand. Could someone please explain to me why the residuals need to have constant variance in layman's terms. Thank you in advance if anyone can help me.

$\endgroup$
1
  • 2
    $\begingroup$ Homoscedasticity affects the standard error of the estimated coefficients. If homoscedasticity, the confidence intervals for the estimated effects might be too narrow and our p values too small. $\endgroup$ Nov 4, 2022 at 5:03

3 Answers 3

0
$\begingroup$

The variance of the residuals is constant.

At each residual (true value - predicted value), the difference is about the same.

Variance is generally: $(x_{i}-\bar{x})^{2}$ which is the difference of value of the observation from the average of all observations, squared, showing you how much all the points differ from their average. With a high value meaning they differ a lot, and a low value meaning that they generally are about the same.

You can kind of think of a graph of these residuals as one with dots representing the residual at each prediction (n predictions = n dots), and for each of those dots, another dot illustrating the mean value of the residuals. So you have 2 dots for each prediction: the residual for the prediction and the mean residual for all predictions. If the dots are all about uniformly the same distance apart from one another at each prediction, then you’ve got constant variance. If the dots are all kinds of different distances apart, they vary a lot, and so you’ve variation (potentially heteroskedasticty).

If the variance of the residuals is constant($var(true value - predicted value)$), or very small in applied circumstances, then the estimator predicts or fits a curve of the data that is constantly about the same amount off for each prediction. If that’s the case, then on average, your model predicts each data point with about the same performance. Your estimator performs about the same at each predicted point (for that data).

So, homoskedastcity means the variance of the residuals is constant. If the variance of the residuals is constant, then your model may generalize the data you have well.

The other assumptions of linear regression, or GLMs, are what make the homskedastic assumption also contribute to having a model that represents your data well.

$\endgroup$
0
2
$\begingroup$

why the residuals need to have constant variance in layman's terms

The assumption of homogeneity of variance is important for inference on the coefficients of the model (which are interpreted as the average effects of the variable on the outcome).

The sampling variance of the coefficient $\beta_1$ (the slope) in a linear regression involves the estimated variance

$$ \mathrm{Var}\left(\hat{\beta}_1\right)=\sigma^2 /\left[\mathrm{SST}_1\left(1-R_1^2\right)\right]$$

If $\sigma^2$ is not constant (e.g. is a function of the predictor), then the variance is misspecified and hence the standard error (and consequently confidence intervals and p values) will also be incorrect.

When homogeneity of variance is violated, there exist methods for correcting the standard errors. See "robust standard errors" or "the sandwich estimator"

$\endgroup$
2
  • $\begingroup$ thank you so much $\endgroup$ Nov 4, 2022 at 14:40
  • 4
    $\begingroup$ @user1059114207 if you think either Dave's or Demetri's post helped you providing clarification, consider upvoting them and accept either of them by clicking the tick alongside the respective post. $\endgroup$ Nov 4, 2022 at 16:35
0
$\begingroup$

Without doing any math, when we calculate p-values and confidence intervals, we are calculating them based on wrong distributions. If we assume constant variance, those p-values and confidence intervals are calculated a certain way to give the properties that we expect them to have, and without constant variance, those properties will not hold.

Consequently, while a (say) $95\%$ confidence intervals is supposed to contain the true value $95\%$ of the time, we might wind up in a position where such a confidence interval contains the true value $90\%$ of the time, maybe $80\%$ or $60\%$ of the time. However it happens, we wind up being more confident than we should be, since what we think is a $95\%$ confidence interval is really a $90\%$ (or $80\%$ or $60\%$) confidence interval.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.