7
$\begingroup$

I was asked if a type I error in the Shapiro-Wilk test would impact the main analysis and if the wrong test was used if it would matter or not if my data was normally distributed...

$\endgroup$
  • $\begingroup$ @a_statistician - might want to expand that a little and post it as an answer, since it is! $\endgroup$ – jbowman Nov 11 '18 at 3:02
14
$\begingroup$

In statistical analysis, if your data follow a parametric distribution, you should utilize the benefit of knowing the distribution, and employ the statistical methods based on that distribution.

But sometimes we do not know the distribution of the random variable, so the nonparametric statistical methods were developed to embrace the wide range of the distributions while sacrificing some efficiency.

Given you know the distribution of random variable and use the nonparametric statistical method, instead of parametric statistical methods based on knowing the distribution, it will be inefficient, i.e., the power of test will decrease, standard error will increase, and the confidence intervals will be wider than with the parametric method.

$\endgroup$
12
$\begingroup$

If your data happened to be drawn from a normal population (and the other usual assumptions for an ordinary t-test apply), then the test works as it should (it's non-parametric, it's supposed to work). There's no drama on that score.

If you know enough that you're confident in assuming normality you may want to take advantage of that knowledge, but for many tests it doesn't help you a lot.

If you're doing one of the common location-tests (Wilcoxon signed rank test, Wilcoxon-Mann-Whitney test) you lose almost nothing (power-wise) in a test for a location shift by ignoring the normality. [You need one extra observation for every 21 observations to match the power of the most powerful test when all its assumptions hold.]

If you're dealing with some other tests is may matter a bit more (though some may matter even less). One example where it makes a somewhat bigger difference is using a Friedman test compared to the corresponding ANOVA test in a randomized blocks design.

$\endgroup$
  • $\begingroup$ Aren't those tests are only location shift tests if the distributes have the same shape (normal here) and the same variance? $\endgroup$ – Alexis Nov 11 '18 at 17:37
  • 1
    $\begingroup$ @Alexis The assumption under the null for a permutation test is exchangeability (so permuting of labels/signs etc as needed doesn't change the distribution of the statistic); typically the slightly stronger "independent identically distributed" is assumed for H0, giving convenient significance level calculations for rank tests. Combined with a location-shift alternative (which is already specified in my answer), this gives the identify of shape and spread you mention. Perhaps I need to write down algebraically what "location shift alternative" means algebraically so people can see it's covered $\endgroup$ – Glen_b Nov 11 '18 at 23:08
  • 1
    $\begingroup$ If we're not willing to specify a location shift alternative (or at the least to say "that's what we're interested in seeing the power against), I'd say we shouldn't be holding up a t-test as a basis of comparison in the first place, since it would also be impacted if we didn't have a location-shift alternative. If we abandon the location-shift part of the comparison, until we then specify a sequence of alternatives we're interested in, we're left with a question that specifies too little to compute power on. $\endgroup$ – Glen_b Nov 11 '18 at 23:10
  • 1
    $\begingroup$ @Alexis Actually, on reflection I think I'll include both some algebra and more explanation; it's important to be clear what is implied by the circumstances already given in the question and answer, unfortunately I can't do it right now. Thanks for your comment, it makes it clear I don't explain enough here. $\endgroup$ – Glen_b Nov 11 '18 at 23:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.