3
$\begingroup$

how to simulate data (in R) to generate , sample values 1) variables with specific correlation values for a particular model AND 2) with predefined regression coefficients? 3) Can we also set the mean and SD in the same process? 4) Also how does one simulate the p value/significance of the variable.

This is for imitating existing models for analysis and teaching purposes

Sorry for not being specific : this is for multiple regression, sample values. I would like to specify the mean and SD if possible (apparently not, I can specify only one in order to specify the regression coefficients?)

Thanks for the help.

Improve this question
$\endgroup$
5
  • $\begingroup$ Do you mean simple regression or multiple regression? Do you mean specific population values or specific sample values for correlation, and the regression coefficients? Did you want to specify mean and SD of the DV or the IV or both? (If both then you won't be free to choose the regression coefficients) Please clarify your question. $\endgroup$
    – Glen_b
    Commented Mar 2, 2015 at 6:58
  • $\begingroup$ "I would like to specify the mean and SD" ... I ask again, of what, exactly? The y? The x's? both? $\endgroup$
    – Glen_b
    Commented Mar 2, 2015 at 7:09
  • $\begingroup$ Is this an exercise for some subject? $\endgroup$
    – Glen_b
    Commented Mar 2, 2015 at 8:20
  • $\begingroup$ We are trying to replicate an existing regression model, mainly to show how the relationship changes when variables are added stepwise, moderators and to explain how the significance of the coefficients changes. Since the model is based out of a theory we heavily draw upon, we thought it would be quite useful to replicate the exact model and sample parameters and play around with the underlying data. $\endgroup$
    – kristen
    Commented Mar 2, 2015 at 8:27
  • $\begingroup$ also does this change if I would like to specify the means and/or SDs of Y as well? $\endgroup$
    – kristen
    Commented Mar 2, 2015 at 9:30

1 Answer 1

Reset to default
4
$\begingroup$

1) For the predictors (independent variables, x-variables) only, you want to:

specify sample means, sds, and correlation matrix

this is equivalent to specifying the covariance matrix and the means

2) you want to specify sample coefficients in the regression

You can also specify the residual standard deviation, which will relate to whichever p-value you're interested in.

Step 1 is already addressed in a number of posts on site, such as

a) Generating data with given sample covariance matrix

(A covariance matrix scaled to have unit variances is a correlation matrix, so that works for both)

b) Tool for generating correlated data sets

There's some mention of R code in at least one of those.

2)

a) put the desired coefficients in $\beta$

b) simulate random normal errors

c) regress the errors on the x's and find the residuals

d) scale the residuals to the desired standard deviation (call the result $r$)

e) calculate $y=X\beta+r$

That's it.

There's a few extra tidbits in this post on simulating ANOVA that carry over to the regression case.

If you want to determine one of the p-values you can back out the required residual variance that would give that p-value from the other statistics.

Improve this answer
$\endgroup$
26
  • $\begingroup$ I see your posts have a lot of useful information. Thanks. On a side note, do you have any suggestions on books/online resources for understanding and getting hands on with R simulations?. $\endgroup$
    – kristen
    Commented Mar 2, 2015 at 8:34
  • $\begingroup$ Not really. If the simulations aren't very complex, in R simulations are often easiest via replicate. Is there something specific you needed to know about? $\endgroup$
    – Glen_b
    Commented Mar 2, 2015 at 9:14
  • $\begingroup$ thanks glen..sorry about that I finally got my accounts merged. To repeat what I have asked before --- also does this change if I would like to specify the means and/or SDs of Y (outcome variable/dependent) as well? $\endgroup$
    – kristen
    Commented Mar 13, 2015 at 14:40
  • $\begingroup$ They're implied by the other inputs. If you choose the intercept, $\beta_0$ consistent with (not equal to!) the mean you want, you can get the mean of $Y$. If you specify the SD of the error term then the conditional SD of Y follows. If you mean the unconditional SD of Y, that is a function of the conditional SD of Y and the $X$-matrix and $\beta$ vector. $\endgroup$
    – Glen_b
    Commented Mar 13, 2015 at 14:48
  • $\begingroup$ Thanks again. And what if the same question was for population values for correlation? How dos that change. We are just trying to understand. Thanks $\endgroup$
    – kristen
    Commented Mar 16, 2015 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.