I am using elasticnet package in R .How to choose lamba value in enet() function and how to get coef. values ..??
fit2=enet(new1,q$mariner,lambda=0.5)
Though i have randomly chosen 0.5 ....but how should I choose the correct value and also how to check the coef of the selected variables
Note: I reccomend using the glmnet package to fit elastic nets. It has a vey efficient implementation, and has become a standard. This is written from the perspective of that package, which uses the notation that has become a standard in the statistical learning literature. If you would rather use elasticnet, see the note below for an important caveat.
There are two parameters in elastic net $\alpha$ and $\lambda$. $\alpha$ controls the relative balance between the lasso and ridge regularization, and $\lambda$ controls the overall level (intensity) of regularization.
$\alpha$ is often chosen based on intent and domain knowledge. Do you want a sparse model with many zero coefficients? Then chose $\alpha$ close to a pure lasso regression. Do you want a dense model with many non-zero but small coefficients? Then chose $\alpha$ close to a pure ridge model.
On the other hand, $\lambda$ is a parameter that is tuned during the fitting of the model. The most common way to do this is with cross validation. In cross validation, you split your training data into many "folds" - partitions into a training and tuning (validation) subset - and fit a a model on each fold, one for each lambda. The various out of sample datasets in cross validation allow you to estimate the out of sample error for each value of $\lambda$, and then a heuristic is used to choose the best value of $\lambda$. Common heuristics are choosing the $\lambda$ that minimizes the out of fold deviance, and choosing the largest $\lambda$ one standard deviation away from the minimizer lambda (standard deviation referring to the estimated standard deviation in out of fold deviances).
Here's a picture of out of fold deviance estimates by lambda:
Each data point on the curve is one value of $\lambda$, the plots tend to look better by $log(\lambda)$ though, so that has become a standard. The error bars span the standard deviation of the estimated out of fold deviance, the variance is the estimates comes from the various cross validation folds. I believe this picture was from a binomial (logistic) model, with $10$ fold cross validation.
If you are using the glmnet package in R, which I recommend over elasticnet, there is a built in function to perform cross validation, it's called cv.glmnet. I highly suggest reading the documentation on this function before fitting your first glmnets.
To get coefficients, use R's coef function. This is a general function that is overloaded to work on most model objects (technically a generic function).
For more details, I highly reccoment the following paper by the pioneers in this field:
Regularization Paths for Generalized Linear Models via Coordinate Descent
Caveat: I checked the documentation for the elasticnet package, it uses a non-standard definition of $\lambda$:
lambda: Quadratic penalty parameter. lambda=0 performs the Lasso fit
This is not the use of $\lambda$ that has become standard in the literature - and I believe is equivalent to what is usually called $\alpha$. Be warned.
There are two questions here so let me start with the less open-ended one about extracting the model coefficients. Firstly, it might be worth noting that there are two parameters that need to be specified for the elastic net model; when calling the enet function, one parameter is specified (i.e. the lambda parameter), and the regression coefficient estimates are calculated for all possible values of the second parameter (i.e. an elastic net regularization path). To obtain the regression coefficient estimates for various point along the EN path corresponding to the chosen value of lambda, use the predict function. In predict, the formal argument s is used to specify that values of the second elastic net parameter, i.e. the points at which we wish to obtain the regression coefficient estimates. But s means different things depending on the value of another formal argument, mode; see the function documentation for the details. Finally, the formal argument type should be specified as "coefficients", as opposed to requesting the fitted model.
The question about selecting the appropriate value of lambda does not have a definitive answer. There are two penalty parameters in the elastic net model; one that penalizes the $\ell_1$ norm and one that penalizes the squared $\ell_2$ norm of the regression coefficients. The lambda parameter specifies the square of the $\ell_2$ norm. So specifying a small value of lambda results in a path that will have lower ridge regularization, while specifying a larger value of lambda results in more ridge regularization. Thus lower values of lambda tend to yield sparser model fits, while higher values of lambda tend to yield denser model fits. Still, this does not give us a way to choose an actual value. If prediction accuracy is the primary goal, then one way to choose the values of the elastic net parameters is to choose the parameters that give the smallest prediction error, as measured on a testing set or through cross validation. So in this scenario, one might choose a sequence of values for lambda, and the find the minimum prediction error along the elastic net regularization path for each choice of lambda, and then choosing the lambda that had the smallest prediction value in its path.
Your choice of lambda is going to depend on your desired results. Choosing close to 1 you will get something that performs more like a lasso regression and closer to 0 gives ridge. This vignette: http://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html is extremely helpful in detailing both. I used it as a walk through the first time I needed with elastic net and it performed very well.
