I want to compare few methods of variable selection. I want to do it using simulations. I'm aware of the fact that it won't be an ultimate answer to question 'Which method is the best one?', but I'm looking just form some hint. To do such simulations I need a method to draw a 'random linear model'. Is there any well accepted algorithm of drawing 'random linear model'? By well accepted I mean method that was used for example in some scientific paper.

I was thinking about following simple approach:
1) Choose $n$ and $k$, which denotes number of observations and number of variables.
2) Generate random matrix $X$ by drawing each element using uniform distribution $(0,1)$.
3) Generate parameters using uniform distribution $(0,1)$.
4) Generate residuals using Normal Distribution $(0,\sigma^2)$ for some fixed and arbitrary chosen $\sigma^2$
5) Calculate $Y=X\beta + \epsilon$


2 Answers 2


One problem with the approach you outline is that the regressors $x_i$ will (on average) be uncorrelated, and one situation in which variable selection methods have difficulty is highly correlated regressors.

I'm not sure the concept of a 'random' linear model is very useful here, as you have to decide on a probability distribution over your model space, which seems arbitrary. I'd rather think of it as an experiment, and apply the principles of good experimental design.

Postscript: Here's one reference but i'm sure there are others:
Andrea Burton, Douglas G. Altman, Patrick Royston, and Roger L. Holder. The design of simulation studies in medical statistics. Statistics in Medicine 25(24):4279-4292, 2006. DOI:10.1002/sim.2673

See also this related letter:
Hakan Demirtas. Statistics in Medicine 26(20):3818-3821, 2007. DOI:10.1002/sim.2876

Just found a commentary on a similar topic:
G. Maldonado and S. Greenland. The importance of critically interpreting simulation studies. Epidemiology 8 (4):453-456, 1997. http://www.jstor.org/stable/3702591

  • 1
    $\begingroup$ +1 Good advice and good references. It's difficult to think of what use a "random" model would have: no method is universal, so one needs to test it on data that are characteristic of the intended areas of application. $\endgroup$
    – whuber
    Feb 2, 2011 at 17:28

To address @onestop's objection to non-correlated regressors, you could do the following:

  1. Choose $n, k, l$, where $l$ is the number of latent factors.
  2. Choose $\sigma_i$, the amount of 'idiosyncratic' volatility in the regressors.
  3. Draw a $k \times l$ matrix, $F$, of exposures, uniformly on $(0,1)$. (you may want to normalize to sum 1 across rows of $F$.)
  4. Draw a $n \times l$ matrix, $W$, of latent regressors as standard normals.
  5. Let $X = W F^\top + \sigma_i E$ be the regressors, where $E$ is an $n\times k$ matrix drawn from a standard normal.
  6. Proceed as before: draw $k$ vector $\beta$ uniformly on $(0,1)$.
  7. draw $n$ vector $\epsilon$ as a normal with variance $\sigma^2$.
  8. Let $y = X\beta + \epsilon$.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.