I am currently examining which of sixteen variables are the most important in predicting a binary outcome. There are 907 observations, so obviously $N$ is much larger than $p$
In the last six months after doing a short course and some reading, I have come to the understanding that older methods for variable selection such as stepwise regression are biased and can lead to incorrect inferences, and that penalised regression techniques are better for such tasks.
However most of the papers and posts on CV that I have read (e.g. here and here) that discuss such techniques (e.g. LASSO and Horseshoe regression) say that they are for high-dimensional data when $p>>n$.
Hence I am starting to wonder if it is appropriate to use regularization techniques when $N>>p$?
In low-dimensional data situations like mine do techniques like penalised regression offer an advantage over standard regression?