LASSO regression when model is known I am very new to regression as I have been reading "The Elements of Statistical Learning: Data Mining, Inference, and Prediction" by Hastie et al. on Standford's website this weekend.
My goal is to determine if I can make use of any techniques in my physics research. I am starting to get a slight feel for the LASSO technique, but wanted some input from someone that really knows the technique. It seems like from my reading regression really works best if you don't know your model a priori, which is half true in my case. I have two "systems" of the following:


*

*System 1: I know the model equation. I have 4 complex valued equations of the form $H_j = \sum C_i X_i P(X) $ j=1..4;(where C are known constants different for each equation, X is my complex parameter space, and P are related to the Legendre polynomials). This is an infinite series in parameters, but I know physically that most terms above i=5 are basically 0. My experimental data then is known to fit 16 equations of the form $H_i*|H_j|^*$; hence the 16 equations are non-linear, and all correlated, and unfortunately currently under-determined (not enough data to get a unique solution).


For this system would LASSO be of any use, or would there be another regression technique to recommend? I currently just use a least squares approach.


*

*System 2:
This is more model based. My input is the hopefully correct output of system 1, my output is then some physical values. I don't know how many physical values I need, or what their values are in all cases.
I believe this case really lends itself to LASSO, but not sure how to apply it. So here for this system, how would one apply it? The LASSO equation is:
$\sum_i (y_i - \beta_0 - \sum_{j=1}^p x_{ij}\beta_j)$ under the constraint of the $\sum_j \beta_j$ being smaller than some user determined value. 


Questions:


*

*In my system how do I interpret the $y_i$, $\beta_j$, and $x_{ij}$ to be? 

*Do I replace the least squares technique with this one, or does LASSO work on top of least squares, meaning, do I use both terms to get my best parameter set? 

*Also, anyone know a good book that might give insight into how to program this into Fortran/C++ code? 

 A: Regarding your System 1:
Even when your model is known up to parameter values it may make sense to use LASSO or some other form of regularization (e.g. ridge regression or elastic net). For example, if you know that the true model is
$$ y = X \beta + \varepsilon, $$
but you don't know the values of $\beta$, the minimum mean squared error (MSE) estimator of $\beta$ will involve some shrinkage, while the OLS estimator will have no shrinkage and will generally yield a higher MSE. (I say "some shrinkage" because there is no analytical solution for an optimal amount of it. That might be part of the reason why we use OLS instead of a minimum MSE estimator.)
Regularization should be especially helpful in situations such as this:

This is an infinite series in parameters, but I know physically that most terms above i=5 are basically 0.

It will help sensibly tune down the effect of the high order members that are basically zero and the estimation variance of which may be boosting the mean squared error.

Regarding Question 2 (if I understand it correctly):
LASSO is a substitue for non-penalized regression (OLS). I don't think it makes sense to somehow run both sequentially, or however I should interpret the question "does LASSO work on top of least squares?".
