I need something clarified and that is when you have non-constant variance, estimates won't be biased but will be a problem when it comes to the S.E. formulas and efficiency. Therefore OLS estimates will be inefficient because they give equal weight to outliers. It makes sense to me that giving equal weight to these outliers will cause problems, but why would it cause problems if these data points are genuine and provide information?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
In general, if you have outliers, they cause bias. Outliers are data that come from some other population than the rest of your data and/or the population that you are trying to model. As a result, your parameter estimates reflect the mixture of the population you are after and some sliver of a contaminating population.
The problem of efficiency is different. You can think of efficiency as being similar to statistical power. The idea of efficiency is that you use the information that is available to you optimally. If you have non-constant variance, different data provide differing amounts of information about the conditional mean of $Y$ at a given point in $X$. Clearly, if you give equal weight to each point, you are not using the information optimally, but if you can allocate the weights in accordance with the amount of information in each point, you can achieve greater efficiency. From a practical standpoint, greater efficiency can mean that you have more power to reject a false null, for example.
Here is a simple simulation, in
Here, I contrast an unweighted regression with a regression that weights the data according to how much information they have to offer about the slope (set of conditional means). Notice that the amount of information is the reciprocal of the variance at a given point in $X$, and that I am not estimating it, for this demonstration, I am using the correct value a-priori. (How well this works when the weights are estimated depends on how good the estimates are.)
To understand heteroscedasticity more thoroughly, it may help to read a couple other answers I've provided on the topic:
It's not that there's equal weight given to outliers that's the issue (if the data are generated according to the correct model, the observations won't really be outliers).
The issue corresponding to what you're talking about is that information of different precision is given equal weight, when more weight should be placed on the information with greater precision (i.e. that lies on average closer to the true line).
But even so, that's an issue of efficient use of information. It's possible that the waste of information (and the noisier estimates that result) may not especially bother you.
There's another problem, however - one of larger concern. This problem to do with standard errors of the coefficients, and hence with inference such as hypothesis tests and confidence intervals.
If you don't properly account for the fact that the variance isn't constant, the standard errors of the coefficients are biased.