When I want to simulate Y coming from the linear regression model, $$Y_i = X_i ^T \beta + \epsilon_i,$$

I can use code like:

x = rnorm(100); beta = 1
y = x %*% beta + rnorm(100, sd = 3)

If I want to make the estimate of $\beta$ "noiser" (increase standard error), I can increase the standard deviation in the second line above.

How can I increase the standard error of a logistic regression coefficient? I can simulate from a logistic regression model with:

x = rnorm(100); beta = 1
nu = x %*% beta        # linear predictor
pr = 1/(1+exp(-nu_1))         # pass through an inv-logit function
y = rbinom(100,1,pr)      # bernoulli response variable
  • 2
    $\begingroup$ nu = x %*% beta + rnorm(100,0,1) would do it, altering the standard deviation as required. $\endgroup$ – jbowman Jan 19 at 0:11
  • 3
    $\begingroup$ @jbowman's approach works well, but--unlike the ordinary least squares case--it renders the model invalid by introducing "overdispersion." Whether that's a problem depends on what you're trying to accomplish with your simulation. $\endgroup$ – whuber Jan 19 at 1:48
  • 1
    $\begingroup$ I think @whuber's comment holds the key to getting a good answer - what is your ultimate objective? Why are you trying to make the estimate of $\beta$ "noisier"? Does bias matter? ... $\endgroup$ – jbowman Jan 19 at 17:38

You could make the $x \times \beta$ values be on average substantially below (or above) zero instead of approx. zero (e.g. subtract 2 from all x values). That results in a more unbalanced data set.

Or you could reduce the number of observations. Or you could add additional covariates to the model that also need to be estimated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.