I am trying to understand the logic behind forward-backward selection (even though I know that there are better methods for model selection). In forward model selection, the selection process is started with an empty model and variables are added sequentially. In backward selection, the selection process is started with the full model and variables are excluded sequentially.
Question: With which model does forward-backward selection start?
Is it the full model? The empty model? Something in between? Wikipedia and Hastie et al. (2009) - The Elements of Statistical Learning, page 60 are explaining the method, but I wasn't able to find anything about the starting model. For my analysis I am using the function
stepAIC of the
Below you can find an example in
R. The stepAIC function automatically prints each step of the selection process in the console and it seems like the selection starts with the full model. However, based on the answer of jjet I am not sure if I have done anything wrong.
# Example data N <- 1000000 y <- rnorm(N) x1 <- y + rnorm(N) x2 <- y + rnorm(N) x3 <- y + rnorm(N) x4 <- rnorm(N) x5 <- rnorm(N) x6 <- rnorm(N) data <- data.frame(y, x1, x2, x3, x4, x5, x6) # Selection library("MASS") mod <- lm(y ~., data) stepAIC(mod, direction = "both")