16
$\begingroup$

Some statistical tests are robust and some are not. What exactly does robustness mean? Surprisingly, I couldn't find such a question on this site.

Moreover, sometimes, robustness and powerfulness of a test are discussed together. And intuitively, I couldn't differentiate between the two concepts. What is a powerful test? How is it different from a robust statistical test?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Power and robustness are orthogonal concepts, even if they are two important properties of a test. It seems to me it would be better to ask two separate questions. $\endgroup$
    – Stephan Kolassa
    Nov 8, 2017 at 7:44
  • 2
    $\begingroup$ We could word robustness as the property of an adapted procedure that's insensitive to violations of some assumptions of its core theory. $\endgroup$
    – Firebug
    Nov 8, 2017 at 10:52

2 Answers 2

Reset to default
22
$\begingroup$

Robustness has various meanings in statistics, but all imply some resilience to changes in the type of data used. This may sound a bit ambiguous, but that is because robustness can refer to different kinds of insensitivities to changes. For example:

  • Robustness to outliers
  • Robustness to non-normality
  • Robustness to non-constant variance (or heteroscedasticity)

In the case of tests, robustness usually refers to the test still being valid given such a change. In other words, whether the outcome is significant or not is only meaningful if the assumptions of the test are met. When such assumptions are relaxed (i.e. not as important), the test is said to be robust.

The power of a test is its ability to detect a significant difference if there is a true difference. The reason specific tests and models are used with various assumptions is that these assumptions simplify the problem (e.g. require less parameters to be estimated). The more assumptions a test makes, the less robust it is, because all these assumptions must be met for the test to be valid.

On the other hand, a test with fewer assumptions is more robust. However, robustness generally comes at the cost of power, because either less information from the input is used, or more parameters need to be estimated.

Robust
A $t$-test could be said to be robust, because while it assumes normally distributed groups, it is still a valid test for comparing approximately normally distributed groups.

A Wilcoxon test is less powerful when the assumptions of the $t$-test are met, but it is more robust, because it does not assume an underlying distribution and is thus valid for non-normal data. Its power is generally lower because it uses the ranks of the data, rather than the original numbers and thus essentially discards some information.

Not Robust
An $F$-test is a comparison of variances, but it is very sensitive to non-normality and therefore invalid for approximate normality. In other words, the $F$-test is not robust.

Improve this answer
$\endgroup$
8
  • 1
    $\begingroup$ Your answer is very clear and easy to understand. I edited by question to ask more about whether a test is powerful as I see you discussed that in your answer. Would you mind explaining what does a powerful test means? $\endgroup$
    – JetLag
    Nov 8, 2017 at 6:03
  • $\begingroup$ I included a short description of how this relates to the power of a test. $\endgroup$
    – Frans Rodenburg
    Nov 8, 2017 at 6:14
  • 1
    $\begingroup$ This is a great answer, just want to add that there are ways to formalize the definition. Some consider a test to be robust if it has both the robustness of validity, i.e. significance level of the test is stable given small departures from the null, and the robustness of efficiency, i.e. power is still good given small departures from the specified alternative; and these qualities can be quantified through the use of influence functions. $\endgroup$
    – Francis
    Nov 8, 2017 at 10:28
  • 1
    $\begingroup$ @Eric, isn't the equivalence only true for two groups? $\endgroup$
    – Sextus Empiricus
    Nov 9, 2017 at 9:14
  • 1
    $\begingroup$ @eric_kernfeld I think Frans is referring to the use of $F$ to compare variances not its use in ANOVA. $\endgroup$
    – mdewey
    Nov 9, 2017 at 9:54
4
$\begingroup$

There is no formal definition of "robust statistical test", but there is a sort of general agreement as to what this means. The Wikipedia website has a good definition of this (in terms of the statistic rather than the test itself):

Robust statistics are statistics with good performance for data drawn from a wide range of probability distributions, especially for distributions that are not normal.

https://en.wikipedia.org/wiki/Robust_statistics

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.