Eliminating observations with big residuals in regression I busy with a regression model that seems to have heteroscedasticity. The model has 6 independent variables and one dependent variable. I did the regression and noticed heteroscedasticity. I then eliminated the observations with very high residuals +/- 5% of the total observations which is 25000. I did the regression again without these high residual observations and found that this new regression does not have any signs of heteroscedasticity. I would like to know: is a process like this acceptable or not?
 A: You ask for a yes-no answer, but answers might vary greatly. I won't be the only member here unwilling to approve (or condemn) what I can't see. 
From this description I'd say that the answer might vary from 


*

*that was brutal and utterly ad hoc, but it is just possible that you got a fair model with poor methods 


through 


*

*we can't tell, because just mentioning heteroscedasticity does give any context to judge what is a good model; we need to see the data and know what model you tried (there is zero formal content in your post, although if I had to guess it's linear regression on the variables as given with no extra complications) 


to 


*

*no; it's unacceptably poor practice to drop observations because they happen to be inconvenient compared with simplistic assumptions that would be nice if true. 


Personally, I go with all of these, but my bottom line is as just given. 
On this information, I would suggest some priorities: 


*

*Heteroscedasticity is arguably a secondary problem to do with error structure; you may need much more emphasis on getting the functional form and estimation method right.  

*Additionally, using transformations or a non-identity link function is a much better way to proceed than dropping some fraction of the data. 
I'd add that as your analysis was demonstrably two-phase, the resulting standard errors and P-values from the second leg don't mean much and should certainly not be reported as if the dataset arrived as you left it. 
