I have two paired samples following normal distributions N(0, $\sigma_1^2$) and N(0, $\sigma_2^2$). Samples represent estimation errors (residuals) of two linear regression models used to predict the same response variable using two different methods/independent variables. I have 30 pairs of residuals, so I would like to apply Wilcoxon signed-rank test to check whether means of absolute values or relative errors are different. Since absolute values do not follow normal distribution, I cannot use t-test or something similar.

I would like to find type II error and statistic power of Wilcoxon signed-rank test. Is there some R function (or any other tool) that can be used? I have found a number of functions for testing the power of tests here http://www.statmethods.net/stats/power.html but I’m not sure could they be applied on Wilcoxon signed-rank test. If there is no built-in function is there some other tool or algorithm to manually calculate error?

  • $\begingroup$ While absolute values aren't normal, if the values had the distribution you said, we could easily construct an optimal test for that situation. However, regression residuals don't actually have constant variance, and they aren't independent. The signed rank test - and the power functions you seek - make assumptions that don't hold. Depending on the number of predictors and the design (the specific arrangement of points in the predictors), the dependence may be small or not so small (and the variances may be very similar or quite different). $\endgroup$ – Glen_b Jun 10 '15 at 9:57
  • $\begingroup$ But if we have good regression model, I thought that residuals should be independent, homoscedastic and normally distributed. If I cannot use Wilcoxon sign rank test how can I compare linear regression models? I would like to avoid a simple comparison of mean values of errors, and have statistic test that checks are mean values different. $\endgroup$ – Jovan MSFT Jun 10 '15 at 18:00
  • $\begingroup$ Residuals are demonstrably not independent nor generally homoscedastic when the errors are both, even if the fitted model is exactly correct. Don't confuse residuals with errors. $\endgroup$ – Glen_b Jun 11 '15 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.