Timeline for Which permutation test implementation in R to use instead of t-tests (paired and non-paired)?
Current License: CC BY-SA 2.5
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 24, 2012 at 16:58 | comment | added | gung - Reinstate Monica | There is a lot of good info here, but it is very strongly worded & mixed w/ some things that don't make much sense to me. Eg, re #3, my understanding of the relationship b/t the z-test & the t-test is that they're essentially the same, but z is used when the SD is known a-priori & t is used when the SD is estimated from the data. I don't see how if the CLT guarantees the normality of the sampling dists this licenses the z-test while leaving the t-test invalid. I also don't think $\neq$ SD's invalidates t if the CLT covers you, so long as an adjustment (eg, Welch–Satterthwaite) is used. | |
| Jan 11, 2011 at 10:22 | comment | added | Henrik | +1 Thanks for the detailed answer. One point: The data is paired. I.e., Each data pair is responses from a single participant on different trial types. We constructed the trial in such a way to expect differences in that way. Furthermore, I have to present the results (i.e., there exists differences) as briefly as possible. So, I have to see how I can use your analysis. | |
| Jan 10, 2011 at 21:13 | comment | added | DWin | We seem to be accepting the notion that the underlying distribution is "similar" to the random instantiation. So couldn't one pose the question: are these both from beta(0.25, 0.25) and then the test would be whether they have the same (non-)centrality parameter. And wouldn't that justify using either a permutation test or Wilcoxon? | |
| Jan 10, 2011 at 18:26 | comment | added | chl | (+1) Nice catch! And very good comments. | |
| Jan 10, 2011 at 17:58 | comment | added | whuber♦ | (+1) Good analysis. You're perfectly right that the results are obvious and one doesn't need to press the point with a p-value. You may be a little extreme in your statement of (3), because the t-test does not require normally distributed data. If you are concerned, there exist adjustments for skewness (e.g., Chen's variant): you could see whether the p-value for the adjusted test changes the answer. If not, you're likely ok. In this particular situation, with these (highly skewed) data, the t-test works fine. | |
| Jan 10, 2011 at 17:21 | comment | added | Mike Lawrence | I'm curious if you'd consider the following an appropriate quasi-visualized approach: bootstrap estimates for the moments of the two groups (means, variances, and higher moments if you so desire) then plot these estimates and their confidence intervals, looking for the degree of overlap between groups on each moment. This lets you talk about potential differences across the variety of distribution characteristics. If the data are paired, compute the difference scores and bootstrap the moments of this single distribution. Thoughts? | |
| Jan 10, 2011 at 17:06 | history | answered | user1108 | CC BY-SA 2.5 |