Perform regression with lot of variables (alternative to stepwise) I hope everyone out here is doing well. I am working towards a linear regression model.
I am starting out with 470 variables , most of them are demographics variables by area (zip code).
My target variable is website traffic by zip code.
Due to high number of variables and lack of time, I quickly ran stepwise regression, but most of the coefficients were not making sense, which made me believe that the model is mis-specified (one of the short coming of step wise regression).
I am planning to try lasso regression next. If this doesn't work I will have to take a manual approach and cherry pick variables to model, which is time consuming.
Is there anything else I could try. I'll appreciate suggestions. Thank you. I am open to share metadata or sample dataset if interested. Thanks
 A: As mentioned in the comments to the question, you are interested in inference, not prediction - that is, you are interested in how the explanatory variables influence the outcome.
Any kind of stepwise procedure, and any other automated method of variable selection (eg the lasso or other regularised procedure) will not work as it cannot account for bias due to confounding or mediation. The only way to do this is by applying knowledge of the variables, and making assumptions where necessary, about the inter-connectedness of all the variables. See this answer for details:
How do DAGs help to reduce bias in causal inference?
The key point here is that it is important to adjust for confounding, and just as important not to adjust for mediators. A mediator for one causal path, may be a confounder for another. So different models are needed depending on what the "main exposure" is. With 470 variables this will not be an easy task. One approach would be to do a factor analysis, to reduce the dimensionality of the dataset to something more manageable, and then use the factors in a regression model. You can combine both steps with a structural equation model (SEM). In order to do this you will need to indentify the factors in advance, that is, by grouping similar variables together.
