I am considering the percentage of failures test of Kupiec (1995). A short description of the test can be found here.

The test statistic follows the chi-squared distribution with 1 degree of freedom. I want to calculate the power of the test and I have found a package in R (pwr), which has a chi-squared test included: http://www.statmethods.net/stats/power.html.

I am struggling to calculate the effect size of the test, which is necessary for calculating the test's power. Let's say I am using 0.05 as the significance level. Do you have any suggestions how to obtain the two probabilities necessary to calculate the effect size?

  • $\begingroup$ You may need to say a bit more about the test. Generally speaking though, effect sizes (and the p(H0) and p(H1) which are occasionally required to calculate them) are determined through prior studies and familiarity with existing literature. You may be in a better position to answer that. $\endgroup$
    – Ian_Fin
    Sep 13 '16 at 13:24
  • $\begingroup$ A short description of the test can be found here: value-at-risk.net/backtesting-coverage-tests $\endgroup$
    – abu
    Sep 13 '16 at 13:32
  • $\begingroup$ Effect size is really something you choose rather than something you calculate or estimate from something. A power calculation is saying "for a null hypothesis that's wrong by this much, I want that much power -- what sample size do I need to get it?" $\endgroup$
    – Glen_b
    Sep 14 '16 at 5:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.