5
$\begingroup$

I would like to simulate t-tests of mean differences. I would like to simulate data based on

  1. pre-specified (desired) mean difference between the variables and
  2. desired t value 3) and p-value

Basically with desired mean difference, t-value and desired p-value. Is it possible to simulate the data and if so how to go about it?. (I am working with R)

$\endgroup$
5
  • $\begingroup$ If you mean an equal-variance t-test, then if you frame this t-test as a regression, you essentially already have an answer to your question in an earlier question of yours. $\endgroup$
    – Glen_b
    Commented Mar 19, 2015 at 1:55
  • $\begingroup$ thank you glen. I was hoping it should be similar but I do not know how to frame a t-test as a regression, in this case when I am testing the difference of means of two variables. A little help and I will look into it. thanks $\endgroup$
    – kristen
    Commented Mar 19, 2015 at 2:44
  • 2
    $\begingroup$ Compare the output of t.test(extra ~ group, data = sleep,var.equal=TRUE) with that from summary(lm(extra ~ group, data = sleep)) (this is a built in data set). Note that the t-statistic and p-value for the coefficient in the regression is identical to that for the t-test. This is no accident. The two models are identical. Note that the regression is on the group dummy that's automatically generated from the factor group. Note also that with a single regressor, my comments about p-value for the overall regression apply equally to the p-value for the coefficient. $\endgroup$
    – Glen_b
    Commented Mar 19, 2015 at 4:53
  • $\begingroup$ Glen, so you mean specifying the "t-value" requires either manipulating the coefficients or S.D's? $\endgroup$
    – kristen
    Commented Mar 22, 2015 at 6:52
  • $\begingroup$ ... and sample size or df, of course (in regression more generally, dependence in X's also matters). In the case of the t-test, the coefficient would be the difference in means, and the relevant SD is the pooled standard deviation. $\endgroup$
    – Glen_b
    Commented Mar 22, 2015 at 8:54

1 Answer 1

1
$\begingroup$

Yes you can. What you want is a power analysis, which will help you find the parameters necessary to make up some data.

See the pwr package in R. Use pwr.t.test(). choose 1) how many samples you want and 2) the p-value (alpha) you're shooting for. set power to 0.5. Take note of the effect size (d) returned in the output.

Next decide on the mean difference you want and divide it by d. That's the standard deviation you want to use when simulating data.

Then use rnorm() to make up your two data sets using all of these parameters.

Then, if you want one data set for which a t-test gives you a certain p-value, you can iteratively generate data sets (use repeat), use t-tests to compare the difference between them, check the p-value, and save the data sets if they meet your p-value criteria. pseudo-code looks like this:

repeat {
  set1 <- rnorm()
  set2 <- rnorm()
  tt <- t.test(set1, set2)
  pv <- tt$p.value
  if (pv < 0.05 & pv > 0.04) {
    break
  }
}

Then you will have two data sets for which the t-test gives you a p-value < 0.05 and > 0.06.

Although it looks based on comments like somebody might have a faster solution.

$\endgroup$
8
  • $\begingroup$ This approach doesn't really work correctly--it is biased towards a lower p-value than specified--and it is much less efficient than directly computing the standard deviation needed to achieve a specified t (or p) value. $\endgroup$
    – whuber
    Commented Mar 18, 2015 at 20:14
  • $\begingroup$ edited to be more specific. $\endgroup$
    – tef2128
    Commented Mar 18, 2015 at 23:10
  • $\begingroup$ thanks. this was very specific. I will report how it is working for me. $\endgroup$
    – kristen
    Commented Mar 19, 2015 at 3:13
  • $\begingroup$ 1) is there a way I can specify the t-value here? A specific value? 2) I have the two means, t-value. I would like to generate data that will satisfy the means and the mean difference t-value $\endgroup$
    – kristen
    Commented Mar 22, 2015 at 11:57
  • $\begingroup$ I can compute the p-value from the specific t-value. But I am not sure of there is a more efficient way. $\endgroup$
    – kristen
    Commented Mar 22, 2015 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.