A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

1
vote
0answers
13 views

Intercept in glmnet

I am trying to fit a regularized logistic regression to my data using glmnet. Using $\alpha=1$ I get a LASSO-regression, which is what I want. My problem is though that I don't know how the intercept ...
1
vote
1answer
35 views

LASSO and related path algorithms

I want to understand why LASSO is called a path algorithm. There are also so many related path algorithms that have sprung out: incremental forward stagewise regression, piecewise-linear path ...
0
votes
0answers
30 views

Compare lasso regression models

I have a continuous outcome variable and several different lasso models to predict the outcome. Something like ...
3
votes
0answers
23 views

Regularization for ARIMA models

I am aware of LASSO, ridge and elastic-net type of regularization in linear regression models. Question: Can this (or a similar) kind of regularization be applied to ARIMA modelling (with a ...
2
votes
0answers
36 views

R: Model selection with categorical variables using leaps and glmnet

I have a linear model containing a few continuous variables and four categorical variables, each represented by 12, 3, 4, and 5 dummy variables respectively. When using model selection criteria such ...
0
votes
2answers
31 views

How to handle NA values in shrinkage (Lasso) method using glmnet

I'm using "glmnet" for lasso regression in GWAS. Some variants and individuals have missing values and it seems that glmnet cannot handle missing values. Is there any solution for this? or is there ...
1
vote
1answer
32 views

Sparsity in Lasso and advantage over ridge (Statistical Learning) [duplicate]

I'm learning about the Statistical learning and in the section comparing Lasso and Ridge Regression it shows that the main difference between these two problems is the way the constraint/penalty is ...
-1
votes
0answers
40 views

How to make correlated variables, uncorrelated?

I have 7 independent variables with 3 observations and they are highly(<95) correlated with each other (each of them) and my dependent variable is head count for 3 years( thus only 3 observations ) ...
0
votes
2answers
62 views

Intercept update in logistic regression lasso using coordinate descent: how is it calculated?

I am trying to figure out how the intercept is calculated for logistic regression lasso using coordinate descent algorithm based on this seminal paper: Friedman, J., Hastie, T. & Tibshirani, R. ...
2
votes
2answers
87 views

Penalizing the Ordinary Least Squares estimation

In a regression analysis, we aim to find the best relationship between two variables (independent variable denoted $y$ and other dependent variable denoted by $x$, and which are related by: $y = ...
1
vote
0answers
30 views

How do you do constrained non-linear least squares in R [migrated]

I am fitting a non-linear least squares model in R. I wish to minimize $(Y - f(Xb))^2$ where $f$ is a nonlinear monotone differentiable function, $X$ is a set of features and $b$ is the parameter ...
3
votes
1answer
27 views

Very low Rsquared of Lasso on Test sample. But very low MSE too?

I am not sure what is going wrong here. I did the following : ...
5
votes
3answers
118 views

Do we still need to do feature selection while using Regularization algorithms?

I have one question with respect to need to use feature selection methods (Random forests feature importance value or Univariate feature selection methods etc) before running a statistical learning ...
0
votes
1answer
48 views

interpreting coefficient values in lasso regression

I am running a lasso regression function. I have about 45 features and I am predicting 1 dependent variable. After running lasso regression I get the coefficient values of the features. 1.If I look ...
0
votes
1answer
17 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust the penalty $\lambda$ in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties ...
0
votes
0answers
25 views

Specification of mixed model structure in glmmLasso

I am having difficulties specifying the appropriate structure for nested/random effects in a mixed model that I am trying to pass through the 'Lasso' shrinkage algorithm. I am using the package ...
0
votes
0answers
34 views

R - quadprog package for constrained Lasso (penalized) linear regression

What I am doing so far: I am doing a constraint linear regression with R's quadprog package, function solve.QP(). The ...
2
votes
1answer
35 views

cv.glmnet - choose lambda to include specific number of variables

I am running LASSO regression selection models using cv.glmnet(). Predicted is the incidence of a disease and I have 63 coviarates to include. Of these 63 ...
0
votes
0answers
38 views

Memory Usage of glmnet with Multiresponse Gaussian Family

I have a large multivariate response matrix that I would like to use to fit an elastic net/lasso model. My $Y$ matrix is $5500 \times 13000$ and my $X$ matrix is $5500 \times 1500$. The $Y$ matrix is ...
1
vote
1answer
48 views

GLM Interaction Lasso

Apparently the stepwise produce in R is not a good way to automatically select the best glm model. Different sources suggest using lasso instead. I had a look at the glmnet packages but I do not ...
6
votes
2answers
64 views

Can $\|\beta^*\|_2$ increase when $\lambda$ increases in Lasso?

If $\beta^*=\mathrm{arg\,min}_{\beta} \|y-X\beta\|^2_2+\lambda\|\beta\|_1$, can $\|\beta^*\|_2$ increase when $\lambda$ increases? I think this is possible. Although $\|\beta^*\|_1$ does not increase ...
4
votes
1answer
53 views

Range of lambda in elastic net regression

$\def\l{|\!|}$ Given the elastic net regression $$\min_b \frac{1}{2}\l y - Xb \l^2 + \alpha\lambda \l b\l_2^2 + (1 - \alpha) \lambda \l b\l_1$$ how can an appropriate range of $\lambda$ be chosen ...
0
votes
0answers
25 views

Getting very small values for lambda with glmnet()?

I'm using glmnet() to analyze a weather data set of 50 variables and 240 observations. My question is pretty simple: when I run ridge regression and LASSO on the ...
7
votes
1answer
74 views

Lasso for explanatory models: shrinked parameters or not?

I'm conducting an analysis where the primary goal is to understand the data. The dataset is large enough for cross-validation (10k), and predictors include both continuous and dummy variables, and the ...
0
votes
0answers
33 views

How can we estimate the predictive interval in Lasso regression

Dear Community members, I am using lasso to solve an inverse problem (a Fredholm) which I can reframe as \begin{equation}\min_{\mathbf x ~~{\rm with}~~x_n\geq 0} \ell_{\rm Lasso}(\mathbf x, ...
0
votes
0answers
50 views

Valid to run glmnet package with single lambda value for lasso regression instead of sequence of lambda values?

I'm executing a few test runs of a lasso regression with the glmnet package in R using the diabetes dataset (http://www4.stat.ncsu.edu/~boos/var.select/diabetes.tab.txt). I’m choosing a single lambda ...
0
votes
0answers
16 views

Library for using LASSO to tune parameters of arbitrary model?

I typically see LASSO applied to the question of finding coefficients of a linear model. I'm looking for a library/tool that performs LASSO with an aim to tune the parameters of an arbitrary ...
1
vote
1answer
71 views

Interpretation of coefficients of glmnet - LASSO/Cox model?

I have done a LASSO / Cox model run for a large dataset of 10K observations which has 1200 Variables. ...
1
vote
0answers
51 views

Regularized regression with missing data?

Are you aware of any regularized regression methods (i.e. Lasso, elastic net) which allows for using cases with incomplete (missing) data (e.g. using EM estimation)? And if yes, is the method ...
5
votes
1answer
36 views

Can I penalize an arbitrary regression model and get Elastic-Net-esque results?

Consider an arbitrary-ish regression model with the unpenalized likelihood $$ \log \mathcal{L} = \sum_i f\left(y_i\,|\,g(\beta_0 + \beta x_i)\right) $$ with $\beta = \left(\beta_1, \dots, ...
4
votes
1answer
93 views

What are the disadvantages of using Lasso for feature selection?

As far as I understand, feature selection is difficult for classification problems because it's effectively impossible to identify an optimal subset of $k$ features in problems where the the total ...
2
votes
0answers
34 views

Heteroskedasticity and skewness in regression, “in general”

In another question of mine, I asked about fitting linear models based on the second-order Taylor expansion: $$ Y = \beta_0 + \sum_i \beta_i (X_i - x_{0i}) + \sum_{i,j} \beta_{i,j} (X_i - x_{0i})(X_j ...
2
votes
1answer
42 views

R Lasso regression for numeric outcome

I do not have experience of using LASSO regression glmnet. I wanted to use it to see which factors affect most my attribute. I have 99 factors (binomial) that affect one attribute (numeric). Is it ...
0
votes
0answers
39 views

Best procedure for evaluating group differences in a Lasso regularized regression

I am evaluating 25 predictors (continuous, ordinal, multinomial) on an ordinal outcome variable using a lasso regularized regression. I am using the lasso for variable selection, to determine which ...
0
votes
0answers
22 views

Hazard ratio can be computed from shrinkage estimators?

I am using LASSO in survival analysis. I wonder if it is valid to compute a hazard ratio (HR) from a lasso coefficient, that is, $\text{HR} = \exp(\text{lasso coef})$? Thank you for any help!
5
votes
1answer
102 views

LASSO closed form with two regressors, JRSSB eq. (6)

I was having look at the orginal Tibshirani paper, JRSSB 1996. In particular, I am trying to understand his equation (6), which says that the LASSO estimates $(\hat\beta_1,\hat\beta_2)$ in the case of ...
0
votes
1answer
88 views

Neural Nets, Lasso regularization

How does one implement lasso regularization or elastic net on neural networks? (feed forward in particular). I know that closed form solutions for this problems don't exist, still how are they ...
1
vote
0answers
65 views

Is it posible to use factor (categorical) variables in glmnet for logistic regression in R?

I'm building a logistic regression in R using LASSO method with the functions cv.glmnet for selecting the lambda and ...
1
vote
0answers
30 views

R package to do a regularized “quasi-binomial” regression

I have data that I want to model with the following data generating process: $y_{i}$~$binomial(p_{i}, N_{i})$ $logit(p_{i}) =\alpha + \beta*X_{i}$ This sort of thing is easily handled in R's glm ...
0
votes
0answers
97 views

How to use LASSO to select glm model gaussian

I have a small sample size n<20. I want to find which combination of 8 variables better predict y. I was using a stepAICc but it is suggested to away stepwise model selection. I have tried lars ...
3
votes
1answer
175 views

Why is there no intercept in the lars output for LASSO in Stata?

This is my first time using lars, so this question is probably obvious. When I run lars on my data I get an output with a model and coefficients assigned to predictors, but there is no intercept. I ...
2
votes
2answers
179 views

Using LASSO for variable selection, then using Logit

I know this would muddy the statistical inference, but I am really only concerned with getting as close to an accurate model as I can. I have a dichotomous outcome variable, with a large set of ...
0
votes
0answers
24 views

Generation of synthetic data for lasso

I am seeking a principled method to generate synthetic data which is an appropriate application for the lasso. More specifically, I want a linear model with sparse coefficients, where the coefficients ...
0
votes
0answers
130 views

LASSO plot label lines with names using glmnet

First, thanks to all of you, this message board is truly helpful, at least up until now;) I used the search option and checked all related questions about LASSO but I could not find an appropriate ...
0
votes
0answers
99 views

Searching for the non-zero coefficients in lasso regression using glmnet.

I have to analyze genomic data set: ~ 22 000 of gene expressions for the two groups each of 40 subjects. I have tried different methods to find genes, which are significantly different among two ...
2
votes
1answer
40 views

How to convert the objective function to canonic form of sparse coding?

As we know the conventional sparse coding problem (LASSO) is: $\min_{\alpha} \| X-D\alpha\|_F^2 + \lambda \|\alpha\|_{1} \tag{1}$ where $X$ , $D$, and $\alpha$ are data, dictionary and coefficients ...
2
votes
0answers
53 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
1
vote
1answer
84 views

Tuning alpha parameter in LASSO linear model in scikitlearn

I'm using the LASSO method, in the problem of text classification (sentiment classification). The features I'm using are mainly Ngrams (every N consecutive words) and I'm using the LASSO specifically ...
0
votes
2answers
137 views

Pointwise convergence in probablity of lasso

In the Knight and Fu's paper, in Equation 6 authors consider the pointwise convergence in probability as $$\underset{\phi \in K}{\operatorname{sup}} | Z_n(\phi)-Z(\phi)-\sigma^2| \longrightarrow_p ...
5
votes
1answer
201 views

Dealing with hierarchical (panel, multi-level) data and fixed effects in LASSO?

The question pretty much explains itself. When running a Lasso regression on a lot of indexed (say by time and location) explanatory variables, is it best practice to transform all data using a ...