A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

0
votes
0answers
27 views

How can we estimate the predictive interval in Lasso regression

Dear Community members, I am using lasso to solve an inverse problem (a Fredholm) which I can reframe as \begin{equation}\min_{\mathbf x ~~{\rm with}~~x_n\geq 0} \ell_{\rm Lasso}(\mathbf x, ...
0
votes
0answers
32 views

Valid to run glmnet package with single lambda value for lasso regression instead of sequence of lambda values?

I'm executing a few test runs of a lasso regression with the glmnet package in R using the diabetes dataset (http://www4.stat.ncsu.edu/~boos/var.select/diabetes.tab.txt). I’m choosing a single lambda ...
0
votes
0answers
11 views

Library for using LASSO to tune parameters of arbitrary model?

I typically see LASSO applied to the question of finding coefficients of a linear model. I'm looking for a library/tool that performs LASSO with an aim to tune the parameters of an arbitrary ...
1
vote
0answers
20 views

Interpretation of coefficients of glmnet - LASSO/Cox model?

I have done a LASSO / Cox model run for a large dataset of 10K observations which has 1200 Variables. ...
1
vote
0answers
35 views

Regularized regression with missing data?

Are you aware of any regularized regression methods (i.e. Lasso, elastic net) which allows for using cases with incomplete (missing) data (e.g. using EM estimation)? And if yes, is the method ...
4
votes
0answers
20 views

Can I penalize an arbitrary regression model and get Elastic-Net-esque results?

Consider an arbitrary-ish regression model with the unpenalized likelihood $$ \log \mathcal{L} = \sum_i f\left(y_i\,|\,g(\beta_0 + \beta x_i)\right) $$ with $\beta = \left(\beta_1, \dots, ...
4
votes
1answer
54 views

What are the disadvantages of using Lasso for feature selection?

As far as I understand, feature selection is difficult for classification problems because it's effectively impossible to identify an optimal subset of $k$ features in problems where the the total ...
2
votes
0answers
22 views

Heteroskedasticity and skewness in regression, “in general”

In another question of mine, I asked about fitting linear models based on the second-order Taylor expansion: $$ Y = \beta_0 + \sum_i \beta_i (X_i - x_{0i}) + \sum_{i,j} \beta_{i,j} (X_i - x_{0i})(X_j ...
2
votes
1answer
28 views

R Lasso regression for numeric outcome

I do not have experience of using LASSO regression glmnet. I wanted to use it to see which factors affect most my attribute. I have 99 factors (binomial) that affect one attribute (numeric). Is it ...
0
votes
0answers
29 views

Best procedure for evaluating group differences in a Lasso regularized regression

I am evaluating 25 predictors (continuous, ordinal, multinomial) on an ordinal outcome variable using a lasso regularized regression. I am using the lasso for variable selection, to determine which ...
0
votes
0answers
16 views

Hazard ratio can be computed from shrinkage estimators?

I am using LASSO in survival analysis. I wonder if it is valid to compute a hazard ratio (HR) from a lasso coefficient, that is, $\text{HR} = \exp(\text{lasso coef})$? Thank you for any help!
5
votes
1answer
92 views

LASSO closed form with two regressors, JRSSB eq. (6)

I was having look at the orginal Tibshirani paper, JRSSB 1996. In particular, I am trying to understand his equation (6), which says that the LASSO estimates $(\hat\beta_1,\hat\beta_2)$ in the case of ...
0
votes
1answer
60 views

Neural Nets, Lasso regularization

How does one implement lasso regularization or elastic net on neural networks? (feed forward in particular). I know that closed form solutions for this problems don't exist, still how are they ...
1
vote
0answers
37 views

Is it posible to use factor (categorical) variables in glmnet for logistic regression in R?

I'm building a logistic regression in R using LASSO method with the functions cv.glmnet for selecting the lambda and ...
1
vote
0answers
21 views

R package to do a regularized “quasi-binomial” regression

I have data that I want to model with the following data generating process: $y_{i}$~$binomial(p_{i}, N_{i})$ $logit(p_{i}) =\alpha + \beta*X_{i}$ This sort of thing is easily handled in R's glm ...
0
votes
1answer
85 views

How to use LASSO to select glm model gaussian

I have a small sample size n<20. I want to find which combination of 8 variables better predict y. I was using a stepAICc but it is suggested to away stepwise model selection. I have tried lars ...
3
votes
1answer
90 views

Why is there no intercept in the lars output for LASSO in Stata?

This is my first time using lars, so this question is probably obvious. When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept. I ...
2
votes
2answers
135 views

Using LASSO for variable selection, then using Logit

I know this would muddy the statistical inference, but I am really only concerned with getting as close to an accurate model as I can. I have a dichotomous outcome variable, with a large set of ...
0
votes
0answers
18 views

Generation of synthetic data for lasso

I am seeking a principled method to generate synthetic data which is an appropriate application for the lasso. More specifically, I want a linear model with sparse coefficients, where the coefficients ...
0
votes
0answers
77 views

LASSO plot label lines with names using glmnet

First, thanks to all of you, this message board is truly helpful, at least up until now;) I used the search option and checked all related questions about LASSO but I could not find an appropriate ...
0
votes
0answers
56 views

Searching for the non-zero coefficients in lasso regression using glmnet.

I have to analyze genomic data set: ~ 22 000 of gene expressions for the two groups each of 40 subjects. I have tried different methods to find genes, which are significantly different among two ...
2
votes
1answer
37 views

How to convert the objective function to canonic form of sparse coding?

As we know the conventional sparse coding problem (LASSO) is: $\min_{\alpha} \| X-D\alpha\|_F^2 + \lambda \|\alpha\|_{1} \tag{1}$ where $X$ , $D$, and $\alpha$ are data, dictionary and coefficients ...
2
votes
0answers
49 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
0
votes
1answer
52 views

Tuning alpha parameter in LASSO linear model in scikitlearn

I'm using the LASSO method, in the problem of text classification (sentiment classification). The features I'm using are mainly Ngrams (every N consecutive words) and I'm using the LASSO specifically ...
0
votes
2answers
135 views

Pointwise convergence in probablity of lasso

In the Knight and Fu's paper, in Equation 6 authors consider the pointwise convergence in probability as $$\underset{\phi \in K}{\operatorname{sup}} | Z_n(\phi)-Z(\phi)-\sigma^2| \longrightarrow_p ...
5
votes
1answer
186 views

Dealing with hierarchical (panel, multi-level) data and fixed effects in LASSO?

The question pretty much explains itself. When running a Lasso regression on a lot of indexed (say by time and location) explanatory variables, is it best practice to transform all data using a ...
0
votes
0answers
53 views

regarding using Lasso and Random forest based on the variable selection result coming from other processes

After the process of data exploration process and discussion with client, we set up a set of variables as follows: T1, T2, T3, T6, T8, T2*T3, T1*t6 During ...
1
vote
0answers
25 views

Why all coeficents of features of model are zero while I have high deviance using glmnet?

I'm using gmlnet to learn lasso regression model. model<-cv.glmnet(x, y, alpha=1, nfolds=10,parallel= TRUE) when I learn model and look at the model it's like this : ...
0
votes
0answers
21 views

input variables with different order of magnitude [duplicate]

I need to build a prediction model based on a data set with 5 different independent variables. The data set looks like as follows. The variables in col4 and ...
0
votes
1answer
69 views

Coordinate descent soft-thresholding update operator for LASSO

I was reading this paper (Friedman et al, 2010, Regularization Paths for Generalized Linear Models via Coordinate Descent) describing the coordinate descent algorithm for LASSO, and I can't quite ...
2
votes
1answer
106 views

R-squared for elastic net

How is the R-squared calculated for an elastic net? How about LASSO? Should be different from OLS, or not? Edit: The main problem is as follows: We have all kinds of fruits like $f_1, f_2, ..., fn$ ...
0
votes
0answers
63 views

Are LASSO regression predicted values also biased?

Since LASSO regression biases coefficients to reduce variance, aren't the predicted values also biased? In my case I am looking at fitted values from a predictive logistic regression model with LASSO ...
2
votes
1answer
93 views

Relation between the tuning parameter $\lambda$, parameter estimates $\beta_i$ and constraint $s$ in LASSO logistic regression

In the context of LASSO logistic regression, I understand that $\lambda$ is the tuning parameter obtained by cross validation. There is also the constraint parameter $s$ ($\sum_{i=1}^p|\hat\beta_i|\le ...
5
votes
1answer
66 views

Why is 'relaxed LASSO' different from LASSO?

If we start with a set of data $(X,Y)$, apply LASSO to it and obtain a solution $\beta^L$, we can apply LASSO again to the data set $(X_S, Y_S)$, where $S$ is the set of non-zero indexes of $\beta^L$, ...
0
votes
0answers
48 views

Cross validation of result from glmnet [closed]

I am trying to experiment with glmnet for building the regression model. The cross validation result is shown in the following figure. Looks like to me that mean-square error is totally out of ...
0
votes
0answers
40 views

questions on glmnet result

I am trying to experiment with glmnet for a data set, which has 41 independent variables is 41. There are 80 data points in total. ...
1
vote
1answer
114 views

Elastic net regularization: mean square error monotonically increases with lambda

This is quite coincidental as my question is nearly identical to this one asked shortly before, but I am also using elastic net regularization with R's glmnet library as a method of variable selection ...
3
votes
1answer
46 views

Is there a way to add inequality constraints on the LASSO in R?

I am trying to use LASSO for model selection, but I need my fitted values to remain non-negative. Is there a way to implement this simply in R? I've found that the penalized package allows for non ...
1
vote
1answer
96 views

How can I implement lasso in R using optim function

As you know lasso is a popular variable selection method of the form of $ (y-x\beta)'(y-X\beta)+\lambda \sum_i|\beta_i| $ the first is that it is possible to use optim() function in R to minimize ...
0
votes
0answers
34 views

Slow Lasso Performance Using sklearn

I am trying to use scikit-learn's LassoCV and/or ElasticNetCV functions to model a dataset with a large (>800) number of ...
3
votes
0answers
61 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
0
votes
0answers
26 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
2
votes
1answer
31 views

Least Angle regression coefficient reaches zero after included

In LARS how is it possible that after including a variable it could reach zero again? http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf I understood that it works like: 1) choose ...
2
votes
2answers
151 views

Ridge & LASSO norms

This post follows this one: Why does ridge estimate become better than OLS by adding a constant to the diagonal? Here is my question: As far as I know, ridge regularization uses a l2-norm (euclidean ...
5
votes
2answers
284 views

Quadratic Programming and Lasso

I'm trying to perform a lasso regression, which has following form: Minimize $w$ in $(Y - Xw)'(Y - Xw) + \lambda \;\text{norm}(w,1)$ Given a $\lambda$, I was advised to find the optimal $w$ with ...
0
votes
2answers
77 views

Coordinate descent on objective function with discontinuous derivative

I'm trying to perform a customized nonlinear regression. I'm using the Linex loss function instead of least-squares. I'm doing LASSO-style regularization, so that my objective function has ...
4
votes
1answer
242 views

How to interpret variables that are excluded from or included in the lasso model?

I got from other posts that one cannot attribute 'importance' or 'significance' to predictor variables that enter a lasso model because calculating those variables' p-values or standard deviations is ...
1
vote
1answer
92 views

Is standardizing data necessary for glasso?

I am wondering, if it is necessary to standardize data (mean zero and stddev eq. 1) for glasso. In many papers on glasso this is mentioned to have data with mean=0 and stddev 1, while using covarience ...
0
votes
0answers
128 views

model selection through sparse group lasso

I am trying to get familiar with the package SGL. The reference is http://cran.r-project.org/web/packages/SGL/SGL.pdf. I typed the example in, and tried to get the coefficients of the fitted penalized ...
0
votes
0answers
31 views

Lasso equivalent estimator with differenziable penalty function

The lasso estimator is define as $argmin_{\beta}~ MSE +\lambda \parallel \beta\parallel_1$ I am wondering if there is an alternative penalty function that is $C^\infty$ and that preserves the sparsity ...