In what order should you do linear regression diagnostics? In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty. 
The question is:  Is there any order to apply these? I mean, do we have to analyze outliers very firstly, and then examine multicollinearity? Or reverse?  
Is there any rule of thumb about this?  
 A: I think it depends on the situation.  If you don't expect any particular problems you can probably check these in any order.  If you expect outliers and might have a reason to remove them after detecting them then check for outliers first.  The other issues with the model could change after observations are removed.  After that the order between multicollinaerity and heteroscedasticity doesn't matter. I agree with Chris that outliers should not be removed arbitrarily.  You need to have a reason to think the observations are wrong.
Of course if you observe multicollinearity or heteroscedasticity you may need to change your approach.  The multicollinearity problem is observed in the covariance matrix but there are specific diagnostic tests for detecting multicollinearity and other problems like leverage points look at the Regression Diagnostics book by Belsley, Kuh and Welsch or one of Dennis Cook's regression books.
A: The process is iterative, but there is a natural order:


*

*You have to worry first about conditions that cause outright numerical errors. Multicollinearity is one of those, because it can produce unstable systems of equations potentially resulting in outright incorrect answers (to 16 decimal places...)  Any problem here usually means you cannot proceed until it is fixed.  Multicollinearity is usually diagnosed using Variance Inflation Factors and similar examination of the "hat matrix."  Additional checks at this stage can include assessing the influence of any missing values in the dataset and verifying the identifiability of important parameters.  (Missing combinations of discrete independent variables can sometimes cause trouble here.)

*Next you need to be concerned whether the output reflects most of the data or is sensitive to a small subset.  In the latter case, everything else you subsequently do may be misleading, so it is to be avoided.  Procedures include examination of outliers and of leverage.  (A high-leverage datum might not be an outlier but even so it may unduly influence all the results.) If a robust alternative to the regression procedure exists, this is a good time to apply it: check that it is producing similar results and use it to detect outlying values.

*Finally, having achieved a situation that is numerically stable (so you can trust the computations) and which reflects the full dataset, you turn to an examination of the statistical assumptions needed for correct interpretation of the output.  Primarily these concerns focus--in rough order of importance--on distributions of the residuals (including heteroscedasticity, but also extending to symmetry, distributional shape, possible correlation with predicted values or other variables, and autocorrelation), goodness of fit (including the possible need for interaction terms), whether to re-express the dependent variable, and whether to re-express the independent variables.
At any stage, if something needs to be corrected then it's wise to return to the beginning.  Repeat as many times as necessary.
