Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4536.0    302.6 395300.0 
Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4964.0    423.8 721700.0 

Welch Two Sample t-test

data:  x and y
t = -0.4777, df = 3366.488, p-value = 0.6329
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2185.896  1329.358
sample estimates:
mean of x mean of y 
 4536.186  4964.455 

Using perm package in R and permTS with exact Monte Carlo

    Exact Permutation Test Estimated by Monte Carlo

data:  x and y
p-value = 0.6188
alternative hypothesis: true mean x - mean y is not equal to 0
sample estimates:
mean x - mean y 
      -428.2691 

p-value estimated from 500 Monte Carlo replications
99 percent confidence interval on p-value:
 0.5117552 0.7277040 

My concern here is that the individual costs are not i.i.d. There are many sub-groups of people with very different cost distributions (women vs men, chronic conditions etc) that seem to voilate the iid requirement for central limit theorem, or should I not worry about that?

     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
     0.0      0.0      0.0   4536.0    302.6 395300.0 
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
     0.0      0.0      0.0   4964.0    423.8 721700.0 

Welch Two Sample t-test
    
data:  x and y
t = -0.4777, df = 3366.488, p-value = 0.6329
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2185.896  1329.358
sample estimates:
mean of x mean of y 
 4536.186  4964.455 

Using perm package in R and permTS with exact Monte Carlo

    Exact Permutation Test Estimated by Monte Carlo
    
data:  x and y
p-value = 0.6188
alternative hypothesis: true mean x - mean y is not equal to 0
sample estimates:
mean x - mean y 
      -428.2691 
    
p-value estimated from 500 Monte Carlo replications
99 percent confidence interval on p-value:
 0.5117552 0.7277040 

My concern here is that the individual costs are not i.i.d. There are many sub-groups of people with very different cost distributions (women vs men, chronic conditions etc) that seem to violate the iid requirement for central limit theorem, or should I not worry about that?

I have a data set with tens of thousands of observations of medical cost data. This data is highly skewed to the right and has a lot of zeros. It looks like this for two sets of people (in this case two age bands with > 3000 obs each):

 Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4536.0    302.6 395300.0 
Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4964.0    423.8 721700.0 

If I perform Welch's t-test on this data I get a result back:

Welch Two Sample t-test

data:  x and y
t = -0.4777, df = 3366.488, p-value = 0.6329
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2185.896  1329.358
sample estimates:
mean of x mean of y 
 4536.186  4964.455 

I know its not correct to use a t-test on this data since its so badly non-normal. However, if I use a permutation test for the difference of the means, I get nearly the same p-value all the time (and it gets closer with more iterations).

Using perm package in R and permTS with exact Monte Carlo

    Exact Permutation Test Estimated by Monte Carlo

data:  x and y
p-value = 0.6188
alternative hypothesis: true mean x - mean y is not equal to 0
sample estimates:
mean x - mean y 
      -428.2691 

p-value estimated from 500 Monte Carlo replications
99 percent confidence interval on p-value:
 0.5117552 0.7277040 

Why is the permutation test statistic coming out so close to the t.test value? If I take logs of the data then I get a t.test p-value of 0.28 and the same from the permutation test. I thought the t-test values wold be more garbage than what I am getting here. This is true of many other data sets I have like this and am wondering why the t-test appears to be working when it shouldn't.

I have a data set with tens of thousands of observations of medical cost data. This data is highly skewed to the right and has a lot of zeros. It looks like this for two sets of people (in this case two age bands with > 3000 obs each):

 Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4536.0    302.6 395300.0 
Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.0      0.0      0.0   4964.0    423.8 721700.0 

If I perform Welch's t-test on this data I get a result back:

Welch Two Sample t-test

data:  x and y
t = -0.4777, df = 3366.488, p-value = 0.6329
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2185.896  1329.358
sample estimates:
mean of x mean of y 
 4536.186  4964.455 

I know its not correct to use a t-test on this data since its so badly non-normal. However, if I use a permutation test for the difference of the means, I get nearly the same p-value all the time (and it gets closer with more iterations).

Using perm package in R and permTS with exact Monte Carlo

    Exact Permutation Test Estimated by Monte Carlo

data:  x and y
p-value = 0.6188
alternative hypothesis: true mean x - mean y is not equal to 0
sample estimates:
mean x - mean y 
      -428.2691 

p-value estimated from 500 Monte Carlo replications
99 percent confidence interval on p-value:
 0.5117552 0.7277040 

Why is the permutation test statistic coming out so close to the t.test value? If I take logs of the data then I get a t.test p-value of 0.28 and the same from the permutation test. I thought the t-test values wold be more garbage than what I am getting here. This is true of many other data sets I have like this and am wondering why the t-test appears to be working when it shouldn't.

My concern here is that the individual costs are not i.i.d. There are many sub-groups of people with very different cost distributions (women vs men, chronic conditions etc) that seem to voilate the iid requirement for central limit theorem, or should I not worry about that?