# Simulation of covariates in regression

I am doing a simulation study on a simple linear regression model to see how well OLS estimator performs. I am planning to use coverage probability to assess the estimation of slope parameter using 1000 replications with sample size of 100

Suppose the model is simple as $y_i=0.5+2*x_i+\epsilon_i$

I am not sure which of the following two options to choose in simulation.

1. simulate all 100 $x_i$ values, say from a normal distribution. Then in each of the 1000 replications, use the same 100 $x_i$ values for generation of $y_i$.
2. In each of 1000 replications, first simulate 100 $x_i$ values, and based on these covariate values, generate $y_i$.

Any help is appreciated.

-
can you be more precise on how you plan to use coverage probability? –  Xi'an Oct 3 '12 at 5:09
Hi, Xi'an, I am planning to calculate the confidence interval based on estimate in each of the 1000 replications, and calculate the proportion of intervals that cover the true coefficient values. I am expecting that the coverage probability should be close to nominal values, such as 95%. –  Jerry Oct 3 '12 at 6:02
To use this coverage probability, I think one assumption should be that estimates in each replication are identically distributed. Then I tend to support option 1, because if x_i values change, the standard deviation of estimate of coefficient will change, and this means these estimates are not identically distributed. But I have not found other evidence supporting that option 1 is better over option 2. –  Jerry Oct 3 '12 at 6:08
Without further information, I would use option 2, because otherwise your coverage probability would depend on an arbitrary sample of $x_i$'s. –  Xi'an Oct 3 '12 at 7:44
@Xi'an: Thanks. But do you agree that using option 2, the obtained estimates for the coefficient cannot be treated as iid? –  Jerry Oct 3 '12 at 16:22
show 1 more comment

To answer your question in comments. If you were doing a bootstrap instead of a simulation the bootstrap would suggest something like option 1.

If you want to accept the assumption that the xs are fixed and known which is an important assumption for applying OLS then option 1 demonstrates the performance when the assumption of known x is correct. But suppose you are interested in seeing the sensitivity to the assumption of fixed x. Then you want to simulate with random errors generated for both x and y. That would not be option 1 but it would not be option 2 either.

-
Thanks for this explanation. I understand now when I should use option 1 and how to do a sensitivity analysis. PS: I also searched for some literature and found that option 2 is used for assessing 'the estimator's performance in a wide variety of settings'. This is stated in 1st paragraph on page 8 of the following paper: projecteuclid.org/… –  Jerry Oct 3 '12 at 21:29

for $i \in \{1, 2, \ldots, 100\}$ let

$$\eta_i = .5 + 2x_i$$

which is the linear predictor and then you can simulate

$$y_i \sim \mathcal{N}(\eta_i, \sigma)$$

If you do $j$ replicates, your question boils down to:

$$y_{ij} \sim \mathcal{N}(\eta_i, \sigma)$$

or

$$y_{ij} \sim \mathcal{N}(\eta_{ij}, \sigma)$$

If you are interested in drawing inference about parameters estimates for a given $\boldsymbol{\eta}$, then I think you want $y_{ij} \sim \mathcal{N}(\eta_i, \sigma)$ where you use the same values for your linear predictor every time (i.e., option 1 in your question).

A simple example

set.seed(10)
x <- rnorm(100, mean = 0, sd = 1)
res1 <- sapply(1:10000, function(i) {
coef(lm(rnorm(100, mean = .5 + 2 * x, sd = 1) ~ x))
})

set.seed(10)
res2 <- sapply(1:10000, function(i) {
x <- rnorm(100, mean = 0, sd = 1)
coef(lm(rnorm(100, mean = .5 + 2 * x, sd = 1) ~ x))
})

rowMeans(res1)
# (Intercept)           x
#  0.5002331   2.0010302
apply(res1, 1, sd)
# (Intercept)           x
#  0.1005374   0.1070732
rowMeans(res2)
# (Intercept)           x
#  0.4995853   1.9987682
apply(res2, 1, sd)
# (Intercept)           x
#  0.09977722  0.10169706

-
Thanks for your comment. To me, drawing inference about parameters for a given eta seems to be a very rare case. Can you give an example? Also, do you think I should use option 2 if I want to evaluate the performance of estimation in a more general predictor setting? –  Jerry Oct 3 '12 at 6:53
This may be all well and good. If you are doing this to learn and don't mind reinventing the wheel more power to you. But the world of robust regression is vastly studied. There is a lot of literature out there that will probably answer your questions without you needing to do simulations. Least squares estimates are known to be sensitive to departures from normality. Issues about leverage, sensitivity to outliers, heteroscedasticity have all been thoroughly investigated and regression diagnostics were invented to notice these problems with your data. –  Michael Chernick Oct 3 '12 at 11:30
@MichaelChernick: I agree that this simulation is not for discovering anything new about OLS and I am aware that OLS is sensitive to outliers. For just doing this simulation, your suggestion is I choose option 1? Do you agree with my interpretation about the estimates from option 1? –  Jerry Oct 3 '12 at 16:35
@Jerry You might want to focus on a given $\eta$ if you are interested in studying various residuals. If you don't do anything differently, I am not sure it makes a big difference either way (see edited response above). –  Joshua Oct 6 '12 at 8:22