A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

0
votes
0answers
17 views

L1-based feature selection, then classification

Does it make sense to use L1-based feature selection to reduce the feature set of a model, then use another L1-based machine learning algorithm to train the model on the selected set of features? For ...
0
votes
0answers
7 views

How to extract interaction pairs from glinternet object? [on hold]

I am using the first time glinternet and wonder how I can extract the selected/important interactions from it. I use the following code: ...
0
votes
0answers
21 views

How to plot quantile regression with LASSO in R? [on hold]

Good day! Please help me with plotting quantile regression with LASSO in R. Here are the codes I used. library(rqPen) y<- read.csv("C:\Users\book1.csv", header=F, col.name=c("WD")) ...
0
votes
0answers
16 views

Prediction performance of OLS and Lasso

I am running a comparison of prediction performance of two model using OLS and LASSO respectively. LASSO estimates are computed from LARS algorithm, AIC and BIC were used in model selection. In ...
1
vote
1answer
51 views

package {glmnet} too many variables with Lasso

I used the glmnet-package to do a regression + variable-selection with Lasso. I had n=100 oberservations and p=200 covariables. I always read that after variable-selection with the Lasso there a ...
0
votes
2answers
31 views

Can someone explain what the foldid argument in glmnet does?

I m trying to determine what alpha to use in my glmnet function, but the help file tells me: Note that cv.glmnet does NOT search for values for alpha. A ...
1
vote
1answer
17 views

Penalized regression with zero-inflated models

I'm currently building zero-inflated Poisson & negative binomial predictive models using the zeroinfl() function from the pscl package in R. Incorporating penalized regressions into my model to ...
0
votes
0answers
36 views

How to interpret LASSO graphs

I was given access to a total of 19 pieces of deceptive and persuasive texts along with a percentage on whether the readers of the said text followed through with the text's demands. The 19 cases ...
2
votes
1answer
19 views

The name of 'Fused' Lasso

As many of you know, the Fused Lasso is one of well known penalized methods, which is introduced by Tibshirani, 2005. However, I don't get to the meaning of how it is called. Could anyone give any ...
1
vote
0answers
25 views

glmnet with quasibinomial in R [closed]

I have a fractional response regression model, which is estimated with R's glm with family parameter equal to quasibinomial. Now, I want to perform model selection via lasso on the model. Is there ...
1
vote
2answers
62 views

How do I use Lasso and elastic net as feature selectors?

I have a data set with 900,000 rows and 8 features. I want to look at the significance of each feature so that I can evaluate whether the features I add are viable or not. One method I am using after ...
2
votes
0answers
32 views

Regularized (LASSO) probit regression

I have a binary variable Y that is a dichotomization of an unknown latent variable, generated by a regression model with normal error. Therefore it makes sense to fit a probit model to Y. R enables me ...
1
vote
1answer
27 views

Relationship between LASSO and MAP

What is the relationship between the LASSO regression and MAP? Is the Bayesian interpretation of the LASSO that x_est is the MAP estimate of x under the prior Laplace pdf? If there is also a ...
0
votes
0answers
12 views

adding additional variables to a lasso linear model

in my data set there are 100 variables. I use a lasso regularization in my linear model and about 20 variables are non-zero. I use crossvalidation and an extra validation data set to assess ...
0
votes
0answers
23 views

combining two sources of knowledge about covariates in logistic regression model

i have a number of variables and a dataset and i want to build a linear regression model with shrinkage like lasso. i have also another information about my covariates on their relation with ...
0
votes
0answers
18 views

linear regression model with variables containing multiple attributes

consider we have explanatory variables with two attributes and we would like to create linear regression model. for example we have variable x1 which has two attributes x1.val1=2 and x1.val2=0.8 ...
0
votes
0answers
22 views

LASSO / lars-package / variable ranking

I would like to write a generic R-code, which produces a vector with variable rankings according to the optimal LASSO-sequence based on Cp-Criterion. Please see my simple example below. ...
2
votes
1answer
100 views

Cross validating lasso regression in R

The R function cv.glm (library: boot) calculates the estimated K-fold cross-validation prediction error for generalized linear models and returns delta. Does it make sense to use this function for a ...
4
votes
2answers
110 views

LASSO - normalization of response variable needed?

I wonder whether the response variable needs to be normalized before LASSO estimation (I am using the lars package in R to perform LASSO estimation). My guess is that only right-hand side variables ...
0
votes
0answers
44 views

How to compute median time from survfit and penalized cox

I'm new to the survival package in R. I used the survfit function to calculate the median survival times for the new data after fitting Cox model with (coxph) to my training data. But when I check ...
2
votes
1answer
78 views

Why discrepancy between lasso and randomForest?

Following are 2 plots, one of lasso using glmnet package and other 2 from randomForest (variable importance) of the mtcars data set assessing variable mpg vs others. In the lasso plot, the blue and ...
0
votes
0answers
32 views

glmnet lasso coefficients - are signs better estimated than magnitudes?

As a glmnet user I have been primarily interested in the coefficients from the optimal lasso fit. When reading up on the method, I seem to remember coming across a statement to the effect that the ...
6
votes
1answer
114 views

Any significance of area under curves in lasso plot?

Following plot is obtained on performing LASSO using glmnet package: Is there any significance of area under the curves (using 0 as baseline) in reporting significance of the variables? Can we say ...
2
votes
0answers
28 views

Is there a clear set of conditions under which lasso, ridge, or elastic net solution paths are monotone?

The question What to conclude from this lasso plot (glmnet) demonstrates solution paths for the lasso estimator that are not monotonic. That is, some of the cofficients grow in absolute value before ...
3
votes
1answer
144 views

What to conclude from this lasso plot (glmnet)

Following is the plot of glmnet with default alpha(1, hence lasso) using mtcars data set in R with mpg as the DV and others as ...
0
votes
2answers
95 views

Advantages and disadvantages of glmnet?

What are the advantages and disadvantages of glmnet with default options? glmnet(X,Y) In which situations it works best and in which situations it can be very ...
1
vote
0answers
44 views

how to combine coefficients of a logistic regression model with existing prior knowledge about covariates?

I am working on developing statistical models for fault-localization. on the one hand, i construct a logistic regression model with these considerations: 1-my dependent(response) variable is program ...
2
votes
1answer
65 views

LASSO with two predictors

I have a question regarding LASSO with two predictors, somewhat related to another one of mine posted here. I am trying to illustrate equation (6) of the original paper by Tibshirani, JRSSB 1996, ...
1
vote
0answers
49 views

Insignificant coefficients in Logistic Regression after LASSO variable selection

I am trying to use the LASSO technique to identify which variables to include in my model. I used cross validation to identify the value of lambda which minimizes the CV error. For this minimal value ...
1
vote
0answers
24 views

Lasso - Representing l1-ball constraint using the penalized formulation [duplicate]

I would like to solve the following problem: $$ \beta = \arg\min_{\beta;\|\beta\|_1 \leq1} \|X\beta-y\|_2^2 $$ which happens to be the constrained formulation of the Lasso, considering only ...
1
vote
0answers
30 views

Intercept in glmnet

I am trying to fit a regularized logistic regression to my data using glmnet. Using $\alpha=1$ I get a LASSO-regression, which is what I want. My problem is though that I don't know how the intercept ...
1
vote
1answer
43 views

LASSO and related path algorithms

I want to understand why LASSO is called a path algorithm. There are also so many related path algorithms that have sprung out: incremental forward stagewise regression, piecewise-linear path ...
0
votes
0answers
37 views

Compare lasso regression models

I have a continuous outcome variable and several different lasso models to predict the outcome. Something like ...
3
votes
0answers
72 views

Regularization for ARIMA models

I am aware of LASSO, ridge and elastic-net type of regularization in linear regression models. Question: Can this (or a similar) kind of regularization be applied to ARIMA modelling (with a ...
2
votes
0answers
91 views

R: Model selection with categorical variables using leaps and glmnet

I have a linear model containing a few continuous variables and four categorical variables, each represented by 12, 3, 4, and 5 dummy variables respectively. When using model selection criteria such ...
0
votes
2answers
87 views

How to handle NA values in shrinkage (Lasso) method using glmnet

I'm using "glmnet" for lasso regression in GWAS. Some variants and individuals have missing values and it seems that glmnet cannot handle missing values. Is there any solution for this? or is there ...
1
vote
1answer
56 views

Sparsity in Lasso and advantage over ridge (Statistical Learning) [duplicate]

I'm learning about the Statistical learning and in the section comparing Lasso and Ridge Regression it shows that the main difference between these two problems is the way the constraint/penalty is ...
0
votes
2answers
81 views

Intercept update in logistic regression lasso using coordinate descent: how is it calculated?

I am trying to figure out how the intercept is calculated for logistic regression lasso using coordinate descent algorithm based on this seminal paper: Friedman, J., Hastie, T. & Tibshirani, R. ...
2
votes
2answers
238 views

Penalizing the Ordinary Least Squares estimation

In a regression analysis, we aim to find the best relationship between two variables (independent variable denoted $y$ and other dependent variable denoted by $x$, and which are related by: $y = ...
3
votes
1answer
49 views

Very low Rsquared of Lasso on Test sample. But very low MSE too?

I am not sure what is going wrong here. I did the following : ...
5
votes
3answers
166 views

Do we still need to do feature selection while using Regularization algorithms?

I have one question with respect to need to use feature selection methods (Random forests feature importance value or Univariate feature selection methods etc) before running a statistical learning ...
0
votes
1answer
73 views

interpreting coefficient values in lasso regression

I am running a lasso regression function. I have about 45 features and I am predicting 1 dependent variable. After running lasso regression I get the coefficient values of the features. 1.If I look ...
0
votes
1answer
28 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust the penalty $\lambda$ in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties ...
0
votes
0answers
30 views

Specification of mixed model structure in glmmLasso

I am having difficulties specifying the appropriate structure for nested/random effects in a mixed model that I am trying to pass through the 'Lasso' shrinkage algorithm. I am using the package ...
0
votes
0answers
81 views

R - quadprog package for constrained Lasso (penalized) linear regression

What I am doing so far: I am doing a constraint linear regression with R's quadprog package, function solve.QP(). The ...
2
votes
1answer
83 views

cv.glmnet - choose lambda to include specific number of variables

I am running LASSO regression selection models using cv.glmnet(). Predicted is the incidence of a disease and I have 63 coviarates to include. Of these 63 ...
0
votes
0answers
57 views

Memory Usage of glmnet with Multiresponse Gaussian Family

I have a large multivariate response matrix that I would like to use to fit an elastic net/lasso model. My $Y$ matrix is $5500 \times 13000$ and my $X$ matrix is $5500 \times 1500$. The $Y$ matrix is ...
1
vote
1answer
69 views

GLM Interaction Lasso

Apparently the stepwise produce in R is not a good way to automatically select the best glm model. Different sources suggest using lasso instead. I had a look at the glmnet packages but I do not ...
6
votes
2answers
71 views

Can $\|\beta^*\|_2$ increase when $\lambda$ increases in Lasso?

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases? I think this is possible. Although $\|\beta^*\|_1$ does not increase ...
4
votes
1answer
74 views

Range of lambda in elastic net regression

$\def\l{|\!|}$ Given the elastic net regression $$\min_b \frac{1}{2}\l y - Xb \l^2 + \alpha\lambda \l b\l_2^2 + (1 - \alpha) \lambda \l b\l_1$$ how can an appropriate range of $\lambda$ be chosen ...