1
$\begingroup$

Suppose that I want to simulate from a regression model, either linear regression or some other type of general linear model. Let $x$ be a categorical variable with 4 categories. I am interested in simulating from this model.

We know that if we fit the corresponding GLM, we create a matrix with 3 columns with binary entries indicating the corresponding category. The estimated regression coefficients represent the effect of that category with respect to the removed entry. For example

$$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x_{i3} + e_i.$$

For simulation purposes, should I use the 4 categories or only 3 of them?

$\endgroup$
  • $\begingroup$ What do you mean by "simulation purposes?" And how does this differ from simply solving the regression model? $\endgroup$ – tchainzzz Jun 14 at 7:22
1
$\begingroup$

You have to include only 3 variables in your example. In generalized linear models, categorical variables are included by removing one of the categories. The interpretation is that the effect of such covariates is with respect to the removed category. Thus, if you include all of them, when you fit the model without one category, you will obtain different values. See the following example in R.

Including all categories in the simulation

set.seed(123)
# Simulation including all variables
n<- 1000
X <- t(rmultinom(n = n, size = 1, prob = rep(0.2,4)))
beta <- c(1,2,3,4)
e <- rnorm(n, 0, 0.25)
y <- X%*%beta + e
# Model fit
mod <- lm(y~X[,-1])
# Estimates
mod$coefficients

Removing one category in the simulation

set.seed(123)
# Simulation removing first category
n<- 1000
X <- t(rmultinom(n = n, size = 1, prob = rep(0.2,4)))
beta <- c(2,3,4)
e <- rnorm(n, 0, 0.25)
y <- X[,-1]%*%beta + e
# Model fit
mod <- lm(y~X[,-1])
# Estimates
mod$coefficients
| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.