2
$\begingroup$

Basically, should any decent regression model overestimate 50% of the time and underestimate 50% of the time (in the limit)?

In a scenario where a regression model outputs a price, which has to be non-negative, would the distribution of the residuals of an unbiased model still be expected to be ~symmetric around 0?

Or does the asymmetry in this price example mean that we expect the residual distribution to have a positive skew, since there's no limit to how much the model can overestimate but there is a limit to how much it can underestimate (due to the non-negativity)? Or does it depend on exactly what model is being used and the assumptions the model makes about its error terms?

If a positively skewed residual distribution is expected, is a simple regression adjustment of the predicted values a viable way to improve model accuracy?

Improve this question
$\endgroup$
2
  • $\begingroup$ 1. What do you mean by "decent" in the first sentence? What's the criterion? 2. Prices are non-negative but ordinary regression fits are not constrained in the way you suggest (which might be a reason not to use regression but that's a different issue to the one you ask about). A regression line can go negative within the range of the data even when all the data are positive. $\endgroup$
    – Glen_b
    Commented Sep 7, 2017 at 12:26
  • $\begingroup$ With your price example are you imagining a constant error variance or are you imagining something where the scale is changing along with the mean? (I ask so that it's possible to come up with an illustrative example that matches your intent) $\endgroup$
    – Glen_b
    Commented Sep 7, 2017 at 12:34

2 Answers 2

Reset to default
1
$\begingroup$

A skewed residuals distribution would imply that your model is biased and keeps over or under estimating. So i think the short answer to your question is yes.
In you example scenario your model would predict the conditional mean price. You want to avoid your model systematically over/under estimating the price which means that the residuals are ideally white noise / normal random variable.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Strictly speaking your model probably doesn't take the bound on price into account, and thus you would expect the residuals for an unbiased model to be symmetric around zero. If you model the problem with this constraint, that none of the regression predictions can ever be below zero, it's likely you'll get an asymmetric error distribution. $\endgroup$
    – bibliolytic
    Commented Sep 7, 2017 at 11:24
  • $\begingroup$ "A skewed residuals distribution would imply that your model is biased and keeps over or under estimating" --- can you explain how skewed residuals implies bias? $\endgroup$
    – Glen_b
    Commented Sep 7, 2017 at 12:19
  • $\begingroup$ what i meant was that if the residuals distribution is highly skewed then that would imply that the model keeps over/under estimating and is a bad model for this data. my point was that the answer to @Taimur's question is yes, ideally you want the residuals to be normally distributed around 0. $\endgroup$
    – davidski
    Commented Sep 7, 2017 at 14:01
1
$\begingroup$

For ordinary linear regression distribution of the residuals of a regression model should be symmetric about 0 since they should have a $N(0,\sigma^2(I-H))$ distribution.

Even the price is non-negative, the residuals still could be symmetric around $0$ since the structural component of the model i.e $Xb$ can be anything (include negative values). So using the ordinary least regression might not cause any bias.

However, if your residual is not symmetric about 0 then it suggests you might not use ordinary linear regression. You may consider generalized linear regressions.

The key is residuals not the value of price(i.e only be positive).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.