I am concerned with simulating data for a linear regression model. I need to control the means, variances, and correlations (covariances) between the predictors and the criterion variable. In addition, I need to be able to vary the explained variances ($R^2$). It is obvious to me that the latter must be a function of the earlier, so at least one correlation (covariance) in $\Sigma$ is perhaps dependent on the choice of $R^2$, where $\Sigma=E((Y,X)(Y,X)^T)$, for centered $X$ and $Y$, is the variance-covariance matrix of all variables.

My plan is thus as follows:

  1. Specify $\Sigma$, means, and $R^2$
  2. Simulate data with these sufficient statistics, e.g. by sampling from the multivariate normal
  3. Check estimated $\beta$ (regression coefficient) vector against population (theoretical) coefficients and use the model for unrelated tests/science.

Hence, my approach does not suggest specifying $\beta$ but letting the coefficients be a function of population $\Sigma$, means, and $R^2$. The reason I need to do this is to attribute some realisitc scale to $X$ and $Y$ (e.g., let $Y$ assume an 'income' scale and give $X$ a realistic scale for years of eduction). Therefore, I specify the sufficient statistics instead of regression coefficients. But maybe there is a better way.

Moreover, I have two specific questions:

  1. Given the population variance-covariance matrix $\Sigma$ of one criterion variable $Y$ and a series of predictors (covariates) $X$ , I would like to calculate the vector of true population regression coefficients. Of course, I could simulate data $X$ and $Y$ and use the OLS estimator, but there should be a direct way to use $\Sigma$ in the estimation of population $\beta$?

  2. Which options are there to specify covariances (correlations) in $\Sigma$ given I need a fixed $R^2$ of a linear regression of $Y$ on $X$? This, to systematically vary the explanatory power of the regression model.

  • $\begingroup$ Check the answer to this question. $\endgroup$ – user603 Jul 28 '14 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.