I have a data set with tens of thousands of observations of medical cost data. This data is highly skewed to the right and has a lot of zeros. It looks like this for two sets of people (in this case two age bands with > 3000 obs each):
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0 0.0 0.0 4536.0 302.6 395300.0
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0 0.0 0.0 4964.0 423.8 721700.0
If I perform Welch's t-test on this data I get a result back:
Welch Two Sample t-test
data: x and y
t = -0.4777, df = 3366.488, p-value = 0.6329
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-2185.896 1329.358
sample estimates:
mean of x mean of y
4536.186 4964.455
I know its not correct to use a t-test on this data since its so badly non-normal. However, if I use a permutation test for the difference of the means, I get nearly the same p-value all the time (and it gets closer with more iterations).
Using perm package in R and permTS with exact Monte Carlo
Exact Permutation Test Estimated by Monte Carlo
data: x and y
p-value = 0.6188
alternative hypothesis: true mean x - mean y is not equal to 0
sample estimates:
mean x - mean y
-428.2691
p-value estimated from 500 Monte Carlo replications
99 percent confidence interval on p-value:
0.5117552 0.7277040
Why is the permutation test statistic coming out so close to the t.test value? If I take logs of the data then I get a t.test p-value of 0.28 and the same from the permutation test. I thought the t-test values wold be more garbage than what I am getting here. This is true of many other data sets I have like this and am wondering why the t-test appears to be working when it shouldn't.
My concern here is that the individual costs are not i.i.d. There are many sub-groups of people with very different cost distributions (women vs men, chronic conditions etc) that seem to violate the iid requirement for central limit theorem, or should I not worry about that?
