# How to compare statistical tests?

There are a lot of statistic tests to investigate if our OLS assumptions are right or wrong. My question : How to know if a test is better than another ?

This study about normal distribution gave us an answer to choose, the power of tests.

Doing my research, I've read something on tests for heteroscedasticity.

There could be différences between results of test which are trying to answer the same question so I need to give me rules on how to choose one of them.

Is the criterion of power the only way to compare between tests ?

Assuming that the Shapiro-Wilk got the higher power, is it the best to test normality in every case ?

Edit

Some readings later, I got to say that @Peter Flom was right on his opinon about "There is no single best test of the assumption of normality of residuals in a regression" because it depends on the type of data / the size etc... But it would be pretty intresting for me (as somebody who's working on econometrics) to make a guide for the following tests for OLS assumptions :

Normality of residuals, Heteroscedasticity, Multicolinearity ...

I'll work on it to get an other method than just using robust statistics

• Power is a good criterion but the problem is that power depends on the alternative hypothesis which is often unknown.
– user83346
Jul 29, 2016 at 9:51

There is no single best test of the assumption of normality of residuals in a regression and many researchers recommend using graphical methods and an intuitive approach.

There was a great quote which I am not quite remembering about how testing the assumptions after performing a regression was like sending a rowboat to test an ocean liner or something like that.

The tests will give a p value and, like all p values, it will be dependent on sample size. But the problems that violations of the assumptions cause are not dependent on sample size; they are dependent on the grossness of the violation of the assumptions.

One approach is to just always use robust methods (e.g. quantile regression or robust regression); another is to use both robust methods and "regular" ones and then compare results to see if there are big differences.

EDIT: Run a robust regression and get the predicted values; run an OLS reg. and get the predicted values. Plot the two sets against each other and against actual values.

• That was George Box, specifically referring to testing equality of variance (using very non-robust tests assuming normality): ... "To make the preliminary test on variances is rather like putting to sea in a rowing boat to find out whether conditions are sufficiently calm for an ocean liner to leave port." -- GEP Box (1953), "Non-normality and tests on variances", Biometrika, 40:(3-4), pp318-335. Jul 30, 2016 at 5:04
• That does not really answer my question but your approach is really interesting. Can you develop your point of vue on "use both robust methods and "regular" ones and then compare results to see if there are big differences" ? Aug 5, 2016 at 9:33
• I edited my answer. Aug 5, 2016 at 10:49
• Thanks Peter. It's all clear for me now. I'll keep your approach in mind from now. Aug 5, 2016 at 12:35