Why can't regression via Maximum Likelihood shrink regression coefficients to zero as in LASSO? Does shrinking coefficients to zero not give higher L-likelihood? Does the answer to my question have to do with the fact LASSO is bayesian with Laplace priors and Maximum likelihood is frequentist?
Maximum Likelihood just like minimizing the square of the sum of residuals has no penalty factor to reduce coefficients to zero as LASSO does. So by definition Maximum Likelihood does not shrink coefficients to zero and LASSO does.
However, you can't automatically deduce that the LASSO model will generate a better fit and prediction of what you are estimating. Very often it does not. There are several problems with LASSO.
First, given its penalty factor LASSO has no way of differentiating between a true causal variable that has a high coefficient and should be selected in your model vs. another variable that has little relationship with Y and has a low coefficient. The LASSO algorithm may randomly and often select the weak variable instead of the strong causal variable. And, that's a big problem.
The second problem is that LASSO can completely dismantle the underlying explanatory logic of your original model. Not only it eliminates at times causal variables and substitute them with weak variables (1st problem); it can even at times change the directional sign of your variable coefficients. That's another big problem. You can visualize this problem by doing a search for Images of LASSO coefficient paths. And, you will see how so many LASSO models have very unstable coefficient paths that truly do not make any sense.
The third problem is that LASSO does not automatically generates better forecasts (out-of-sample predictions). Very often it does not. You can visualize that by doing a search for Images of LASSO MSE graphs. These graphs have MSE on the Y-axis with penalty factor on the X-axis. And, many LASSO models out there show a rapidly rising MSE the minute the penalty factor increases. This means that the original model with a penalty factor of Zero was much better at predicting than any of the LASSO models. This situation is surprisingly frequent.
I'll add this: That the LASSO (and associated with it, ridge regression) have Bayesian interpretations is a glimmer of light in the problem of penalization and inference. However, it isn't as easy as "pick priors so that some covariates are shrunk to zero". In order for valid inferences to be made from a bayesian linear model, your priors have to actually reflect your present uncertainty about the phenomenon. Else, the uncertainty yielded in the posterior does not correspond to anything, and your just computing expectations for the sake of doing math.
There is no answer to "How do I select variables for inference". Inference presupposes you have a well defined question to assess. Without that question, what are you really trying to infer?