What is "step size" in LASSO regression? I'm looking at this example for LASSO regression in R: http://machinelearningmastery.com/penalized-regression-in-r/.
It mentions "step size" and "best_step".  And, the documentation here also mentions this idea of a "step".  
I have never heard of a step size being applied to LASSO regression.  Can someone please help me understand where this is coming from?
 A: The LASSO procedure, as implemented in R, produces a family of models parameterized by "penalty" or "λ". This, when plotted against the coefficients of your explanatory variables, goes from a model with all coefficients 0 (excepting the intercept if present), all the way to the unpenalized model . 
Step size is how λ changes between each calculation of the model.
"Best" step is just labeling the chosen model from amongst the family of models. The example uses minimum RSS as the metric for best.
A: They're misusing the word "step" here: they mean to refer to a "step number," not a "step size." In fact, it says as much in the documentation you link to (emphasis mine):

Mode="step" means the s= argument indexes the lars step
             number, and the coefficients will be returned corresponding
  *          to the values corresponding to step s.

As for what steps have to do with Lasso regression, it's because a stepwise algorithm called LAR (Least Angle Regression) is being used to trace out the Lasso solution path. However, the step size is specified according to a specific formula; you can find details in the pleasantly readable original paper, Efron et al., (2003), "Least Angle Regression".
A: I suggest having a look at An Introduction to Statistical Learning book. There, on page 255, you can see that the argument "s" is just the value of $\lambda$. In the case of the example from the book, this value was chosen based on cross-validation. Logically, step size would be a coefficient used in the underlying optimisation algorithm implemented with the glmnet() function in R (think of it as something similar to $\gamma$ in gradient descent), so a bigger step size in lasso would mean that global optimum for $\lambda$ could be found faster but also that this optimum could be missed. Therefore, step size should be chosen wisely.
