What is the relationship between regression analysis, LASSO, and coordinate descent? I'm a complete newbie and trying to understand what exactly LASSO is, how coordinate descent is used with LASSO, and how all of that factors into regression analysis.  I'm totally confused about the interplay between these three things.
The key thing I'm really trying to clarify is terminology - from my reading, it sounds like LASSO is a "technique" or "method" for determining "regression coefficients" for generating a "model" of data.  But LASSO has to be "solved" to find the regression coefficients, and one way to solve LASSO is to use a coordinate descent "technique" or "method".  As I'm sure you can tell by now, I'm not clear on what output coordinate descent gives you, how that output is used in LASSO, and how LASSO is then used to perform regression analysis to generate a model of data.  
Assume I have literally zero statistics training and this needs to be explained to a five year old.  Any and all clarification would be greatly appreciated.
 A: Regression analysis is a statistical process for estimating the relationships among variables. For example, estimating the relationship the weather (input variable) has on stock market (output variable). Multivariate regression is when you want to estimate the effects of several input variables on one output variable. For example lets say you simultaneously wanted to estimate the effects that the weather, national unemployment, interest rate, and price of milk all have on the stock market. 
LASSO regression is used when you are doing a multivariate regression problem and you want to know which of your input variables have an actual effect on the output variable - i.e. you want to estimate a sparse model. The advantage of LASSO over normal regression is that it will theoretically set the coefficients of all "non-relevant" variables to 0. In ordinary regression, due to noise and limited data, even if 1 of the input variables is "non-relevant", the coefficient assigned to it will probably be small, but not not 0. 
In our above example, lets say weather had no effect on stock prices. In LASSO regression, its coefficient will be 0, while in normal regression you might get something like .0123.
This may not sound like a big difference, but in the age of big data, where we can have problems with 1000s of input variables, LASSO regression becomes extremely useful to see which inputs are casually related to the output. 
This brings us to coordinate descent. LASSO regression just means finding the coefficients to minimize the LASSO cost function. This requires a specific algorithm/method. The method used to do this is coordinate descent. This is how the behind the curtain magic of LASSO happens. Note, there is nothing magical about this method: you could in principle get the same estimates by trying all combinations of numbers and just choosing the ones which give you the lowest cost function. Coordinate descent just accomplishes the same thing in a quick and accurate manner. If you are just using a statistical package to get the LASSO estimates, this isnt so important for you since the package implements this algorithm for you.  
