A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

learn more… | top users | synonyms

0
votes
0answers
5 views

Difference bewteen trace plot and cv.glmnet

I am using glmnet to perform lasso logisitc regression. The picture shows the zoomed trace plot to illustrate when the first coefficeints pop out, when relaxing lambda. One can see that this happens ...
0
votes
0answers
8 views

When to use Group lasso over lasso?

Two cases: When should numerous numerical predictors be grouped? is it just based off some theoretical knowledge on the predictors? When should levels(>2) in a factor be grouped together?
0
votes
0answers
17 views

optimism-corrected regression coefficients?

I used a regularized (LASSO) cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. I am ...
3
votes
2answers
72 views

AICc is picking overly complex models - something stricter?

I'd like to know if there are stricter alternatives to automated model selection than AICc / AIC / BIC. We have approximately ten thousand curves, and for each we'd like to find the most parsimonious ...
1
vote
0answers
35 views

Why use group lasso instead of lasso?

I have read the that the group lasso is used for variable selection and sparsity in a group of variables. I want to know the intuition behind this claim. Why is group lasso preferred to lasso? Why ...
2
votes
0answers
29 views

When does LASSO fail? [duplicate]

When does LASSO fail in terms of: out of sample forecasting? selecting relevant variables? (If there is a distinction between 1 and 2 to begin with.) Also, Under what conditions would one be ...
2
votes
1answer
55 views

Lasso will not remove correlated variables

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
1
vote
0answers
31 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
0
votes
2answers
116 views

Lasso for “cherry picking”

I use Lasso logistic regression in order to identify a smaller subset of important variables. I start with N=51 (28/23) and 32 predictors. So far it looks pretty promising, because i can identify ...
-1
votes
2answers
47 views

lasso regression on top of random forest

Some time ago, I found a paper describing usage of lasso/elastic net regression on binary variables come from random forest. In short, (i,j)-th variable takes 1 if given observation belong to leaf no. ...
0
votes
1answer
59 views

Why would LASSO not shrink irrelevant features to zero?

Assume I have 10 features to predict an outcome and I use LASSO regression. Let's say the RMSE of the test set is 20. Now, I introduce 5 more features and predict the same outcome, and I also use ...
0
votes
1answer
31 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
0
votes
1answer
10 views

constructing random effects design matrices for lassop{MMS}

I'd like to use elastic net regression for coefficient estimate and parameter selection on a data set that includes nested structure. I've been experimenting with lassop{MMS} to do so. I'm not a ...
0
votes
0answers
10 views

Does the variance inflation factor make sense for regularized regression?

I have a logistic regression model fit using L1 regularization. There are two variables that entered the model that have a correlation of over 0.90. The VIF for these variables are each about 60 which ...
3
votes
1answer
202 views

Ridge/Lasso Lambda greater than 1

I ran Ridge and Lasso regressions using an algorithm to automatically find the optimum lambda. However, the algorithm couldn't find an optimum lambda between 0 and 1. In some cases I could find ...
3
votes
1answer
53 views

Cross validation after LASSO in complex survey data

I am trying to do model selection on some candidate predictors using LASSO with a continuous outcome. The goal is to select the optimal model with the best prediction performance, which usually can be ...
4
votes
1answer
93 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
1
vote
0answers
31 views

Plogit: Lasso logistic regression with Stata

I want to execute a Lasso logistic regression with Stata. Why? Because... 1.) I have a dummy dependant variable (=> Investment success (1) and failure (0)); samples(1/0)(28/23). 2.) I have a set of ...
0
votes
1answer
9 views

Supplying the number of lambdas in cv.glmnet

The cv.glmnet function states 2 options for its lambda parameter. First is NULL, and then ...
3
votes
1answer
98 views

How is L1 regulaziation derived?

I understand the basic idea of regularization. I am very curious to know the derivations behind it so that I get the complete picture. I though a good place to start leraning about regularization is ...
0
votes
0answers
6 views

convergence of coordinate descent applied to lasso

When using coordinate descent for solving a lasso regression, does normalizing the features impact the convergence rate?
0
votes
0answers
21 views

Interpreting results from lasso regression?

I have a time series data set with about 2million observations and 31 variables, which I break to a few thousand using threshold value for my dependent variable. I am using lasso regression in R to ...
1
vote
0answers
41 views

Autoencoders & Predictive sparse decomposition (PSD) & Alternating Direction Multiplier Method (ADMM)

I am studying Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville. In Chapter #14 Autoencoders the authors write Internally, it has a hidden layer $h$ that describes a code used to ...
2
votes
2answers
56 views

Lasso and Ridge tuning parameter scope

In ridge and lasso linear regression, an important step is to choose the tuning parameter lambda, often I use grid search on log scale from -6->4, it works well on ridge, but on lasso, should I take ...
0
votes
0answers
7 views

Multiple variables / continuous outcome / model formula

I have a set of continuous / discrete variables with which I want to model a continuous outcome. How can I know which type of curve would be a good choice and, therefore, which kind of function to ...
0
votes
0answers
7 views

Iterating Lasso

Can Lasso regression be performed multiple times to systematically clean/remove parameters from a model? Would there be downsides to doing so/would it be considered poor practice? Thanks!
0
votes
0answers
8 views

Unbalanced binary features in LASSO regression

I have a target $y$ that I want to predict from variables $x_1, x_2, \ldots x_k$. Suppose the first of these variables, $x_1$, is a binary variable (i.e., only taking on one of 2 values). If I use ...
0
votes
0answers
8 views

Lasso eliminating features but then they reappear?

Does there exist a set of samples and outputs such that running lasso with some value $\lambda$ for the $\mathcal{L}_1$ penalty zeros out some coefficient $w_i$ but for some $\lambda' > \lambda$, ...
1
vote
1answer
34 views

Selecting a loss-function for k-fold cross-validation over shrinkage parameter

I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $\lambda$ on the basis of cross-validation (CV). In this case, I have 50k ...
0
votes
0answers
41 views

Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
3
votes
0answers
36 views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
0
votes
0answers
27 views

Obtain lasso regression coeficient based LS when $X'X = I$

I need to obtain coefficients of lasso regression based in coefficients of Least Square regression method when $X'X = I $. any help will be appreciated.
0
votes
0answers
12 views

fast way to train a classifier on different but overlapping features

I am training a linear classifier repeatedly on different set of overlapping features. I have a 3D grid of features, each time features from a small sphere from a grid are used to train a classifier, ...
0
votes
0answers
46 views

Can L1 linear regression perform worse than vanilla linear regression on fewer features?

I have a data set with 2 features and I'm trying to predict one real-valued variable. I use linear regression and I measure the error using 10-fold CV and absolute mean error as a metric. I noticed ...
1
vote
1answer
35 views

What is the relationship between regression analysis, LASSO, and coordinate descent?

I'm a complete newbie and trying to understand what exactly LASSO is, how coordinate descent is used with LASSO, and how all of that factors into regression analysis. I'm totally confused about the ...
6
votes
1answer
127 views

LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
0
votes
0answers
23 views

weighted lasso/enet using lqa package in R

I'm using lqa to solve my lasso model. However, I need to define a penalty weight/factor which ranges between 0 and 1. 0 shows no penalty has to be applied and 1 means that the lasso penalty has to be ...
0
votes
0answers
23 views

elastic net regression with hierarchal constraints for higher order interactions

I've been using heirNet and glinternet to preserve hierarchy constraints for pairwise interactions in my model, does anybody know if these methods can be extended to 3 way (or higher order) ...
1
vote
0answers
46 views

Lasso Regression - model predictions are not correct. low r-squared

I am attempting to use Lasso to choose the best variables from a set of 20. I have managed to construct a model using LassoCV, however when using the test data to compare the predicted returns to the ...
0
votes
1answer
47 views

Lasso Regression - Finding multiple candidate models

I have 20 predictors and I am attempting to find several candidate models to then test. I am using the LassoCV library, my following code provides me with the alpha and co-efficients of a model. ...
1
vote
1answer
48 views

Standardized LASSO in R still has intercept

I understand the need to standardize variables when performing LASSO in R (I'm specifically using cv.glmnet, and setting ...
1
vote
1answer
41 views

cross validation after lasso

I used cross validation to select lambda. Then I performed lasso and get non zero coefficients (features). Shall I perform cross validation for these non zero coefficients as a kind of validation?
3
votes
1answer
40 views

Interpreting Special Case for Ridge Regression and the Lasso

The below text is from Statistical Learning Page no.225 Consider a case with $n = p$, and $\mathbf{X}$ a diagonal matrix with 1’s on the diagonal and 0’s in all off-diagonal elements. To simplify ...
8
votes
1answer
190 views

Variable coefficient rises, then falls as lambda decreases (LASSO)

I am regressing a continuous predictor on over 60 variables (both continuous and categorical) using LASSO (glmnet). In examining the variable trace plot, I notice that as log lambda increases, one of ...
1
vote
0answers
29 views

Defining Importance of variables in regression and variable selection

When doing variable selection, one of the most asked questions is which variables are most important, or rank the variables in order of importance. Typically in linear or logistic regression, the ...
0
votes
1answer
39 views

Ordinal outcome survey regularization and variable selection

I am analyzing some survey data with many thousands of respondants. My main dependent variable of interest is a three-level Likert scale (very/pretty/not-so-much). I have ~50 predictors. I would like ...
2
votes
0answers
32 views

Is it possible to use LASSO regression with multi-levlel data?

I have real-time monitoring data where participants report on a variety of variables four times per day for a month. Is it possible to use LASSO regression (e.g,. glmnet r package) with this data? I'm ...
0
votes
0answers
26 views

Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
4
votes
1answer
73 views

How to perform non-negative ridge regression?

How to perform non-negative ridge regression? Non-negative lasso is available in scikit-learn, but for ridge, I cannot enforce non-negativity of betas, and indeed, ...
0
votes
0answers
20 views

Difference of feature importance from Random Forest and Regularized Logistic Regression

I have 13 features in a classification task and I use Random Forest, L1 logistic regression and L2 logistic regression for as separate classifiers and would like to compare their performance. Although ...