I was asked about doing a power analysis of a seasonal kendall trend test. I feel like that would be really difficult to do and I haven't found any documentation or software on how to do it. Is there some way to estimate the required sample size using regression? Hopefully someone can point me in the right direction. Thanks!
1 Answer
The first thing with power is to make sure you have clear specification on exactly the alternative (and assumptions) you're checking power at. This determines the sampling situation under which you evaluate the rejection rate
As I suggest in this answer, simulation is a simple way to approach questions on power.
In your case, you want to specify the rejection rate and compute an $n$. If you have the time (it's usually a matter of minutes) you can compute power across a range of sample sizes right at the start.
At a given $n$, the basic approach to power calculation is (as a kind of pseudocode):
0: choose the circumstances under which you evaluate power (alternative + assumptions)
1: repeat nsim times:
generate a sample of size n under the alternative
compute the test statistic, T
if (T <= Tcrit) increment the count of rejections
rejectrate = rejections/nsim
When you evaluate for many different $n$, step 0 doesn't change, but step 1 is repeated for each $n$ you want to look at. You can use search methods (i.e., root-finding - for example, binary search might be suitable for a Kendall's tau-type statistic) to converge on the required sample size for a given power.
The answer I pointed to before has discussion on computing standard errors of the power estimates, and if you consider smoothing across multiple sample sizes it has some suggestions about ways to approach smoothing rejection rates.
(It may be worth considering working on the scale of $\sqrt n$ for some of these tasks, e.g. it might be a more suitable scale for things like binary search or smoothing across multiple $n$.)
-
$\begingroup$ Thanks Glen_b. I figured the answer would be simulation. I did have a follow up question though. If the critical effect size in this power analysis were a sen slope estimate of 0.5, I am having a hard time wrapping my head around simulating data with a sen slope of 0.5. Is there an easy routine in R that could do this? I'm thinking I would need to make an assumption about the type of increase (linear or exponential) etc. $\endgroup$– JimjCommented Jan 7, 2014 at 16:28
-
1$\begingroup$ Ooh, good question, and I see why that would seem tricky. You would specify all the distributional particulars, including all parameter values (though the power won't depend on all of them). That is, you would specify a model for data which includes some specific parametric form of seasonal trend and find the power at that set of assumptions. If you change the parametric model, the power is likely to be different. $\endgroup$– Glen_bCommented Jan 7, 2014 at 23:14
-
1$\begingroup$ If you want some idea of power at some particular value of Kendall correlation (or at some Sen slope), you can do it with a variety of more or less plausible parametric models. If they all give the same answer, you can probably work out why it doesn't depend on the model. If they give different answers, that's a rather important thing to know. $\endgroup$– Glen_bCommented Jan 7, 2014 at 23:17