0
$\begingroup$

I test normality of 12 residuals that are obtained from resid(lm(...)) command via shapiro.test. Thus I know variance, mean, median etc.

The test don't reject the normality hypothesis. Hence I ask: "what is the power of the test?". Obviously I provide very little data therefore the power is very low. But how low precisely/ approximately?

For some other tests such as t-test there are plenty of online tools or the R function power.t.test. What function/tool should I use in order to compute the power for Shapiro-Wilk test?

$\endgroup$
3
$\begingroup$

Thus I know variance, mean, median etc.

Well the mean of residuals is 0, but you only have sample variance, sample median, etc.

In addition, residuals are not independent, so the assumptions of the Shapiro-Wilk test are violated; with only 12 observations, the error df of the regression is even lower (10 or fewer, presumably); this probably shouldn't be ignored unless you account for the effect of it.

Hence I ask: "what is the power of the test?".

Against what alternative, exactly?

For some other tests such as t-test there are plenty of online tools or the R function power.t.test.

Yes, to which you must supply enough information to identify a precise alternative in order to get the power.

What function/tool should I use in order to compute the power for Shapiro-Wilk test?

You could use simulation, once you specify your alternative. Simulation would also let you take account of the dependence in your residuals.

For testing the distribution of ordinary data, see discussion of an example here

In that answer, results from a sequence of increasingly skew alternatives (gamma distributed) are used to obtain a power curve. Different alternatives would produce other values of power.

In your case you need to simulate the (zero-mean) errors, fit the regression and run the test on the residuals (you can fit the regression directly to the errors themselves, the residuals will be the same as if you constructed simulated data from the model).


However, I don't think it makes sense to use a formal hypothesis test in the first place.

See the second half of this answer for some discussion of why. More details in the later part of this answer.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I understand your answer. I test it in order to analyse the quality of my model. The residuals (as far as I'm concerned) should be normally distributed. $\endgroup$ – petrbel May 15 '15 at 15:17
  • $\begingroup$ how does failure to reject tell you anything about model quality? It certainly doesn't imply the residuals are normally distributed. (I've made an edit to include a link which addresses my concerns in more detail) $\endgroup$ – Glen_b -Reinstate Monica May 15 '15 at 15:22
  • $\begingroup$ So the normality of residuals says nothing about the model? I've seen it frequently employed - for example ppsw.rug.nl/~boomsma/apstatdata/Regrdiag_R.pdf page 6 $\endgroup$ – petrbel May 15 '15 at 15:26
  • $\begingroup$ You seem to have misunderstood the point. It's not that the normality of residuals says nothing, it's that the p-value from the test doesn't measure the degree of non-normality. I agree it's frequently done, but that doesn't mean it makes sense to do so for model-checking. You want to know how badly your inference is affected. It may then be useful to have a measure of how non-normal your data are (an effect-size of sorts). The p-value isn't an effect size. I've added another link which discusses that (it is talking about ANOVA, but the comments carry over to this case). $\endgroup$ – Glen_b -Reinstate Monica May 15 '15 at 15:30
  • $\begingroup$ aha, in that case I understand - thank you for the clarification :) $\endgroup$ – petrbel May 15 '15 at 15:44
3
$\begingroup$

There is no straightforward way, unless you are make an assumption about the way in which your data is non-normal.

When you do a power calculation you start by assuming an alternate hypothesis to be true instead of the null hypothesis. With the two-group t-test you might want to test against "no effect" and assume a real effect size of 0.5.

Now given the assumed effect size you also know the sampling distribution of your t-statistic and therefore how likely it is that it will fall into the reject region.

However, when testing normality the alternate hypothesis is that your variable is not normally distributed. This is not enough to make a power calculation possible. Obviously there is an underlying statistic, but assuming "not normal" is not enough to determine the sampling distribution.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ only 12 observations is probably very low power ... unless the interesting alternatives are very far from normal. And then, probably you should be doing things differently, anyhow ... $\endgroup$ – kjetil b halvorsen May 15 '15 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.