# Are penalized regression techniques greedy algorithms?

In other words, is it feasible any of the various penalized regression techniques (such as ridge regression, lasso, and elasticnet) could completely miss the optimal solution for a regression model due to poorly chosen initial values for the model parameters?

• For what it's worth, I'm not sure greedy algorithms are necessarily sensitive to initial conditions or always miss optimal solutions. My impression is that some min-spanning-tree algorithms and Huffman coding are both greedy and optimal. Still an interesting question though! Feb 15, 2013 at 22:37

• @RobertF everyone and their grandmother has come up with a different way of conducting the optimization, and they are all iterative in nature (like Newton-Raphson, although that involves getting a Hessian and the $L_1$ penalty is not differentiable). For the LASSO, for example, the LARS algorithm can be used to get the trajectory of the parameter values as the penalization term is varied. These methods all go to a local minimum, but if everything is convex then there is only one local optimum, and it is the global optimum.