A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

learn more… | top users | synonyms

1
vote
0answers
6 views

Bias of the Lasso estimator from Tibshirani

I am searching for a Theorem that gives upper bounds for the bias of the Lasso estimator from Tibshirani. Do anybody know such a Theorem? Thanks a lot for help! Best regards, Markus
7
votes
1answer
48 views

What are the pros and cons of employing LASSO for causality analysis?

It looks like social sciences are impressed by Statistical Learning and its results. A couple of months ago, I heard Imbens saying: "LASSO is the new OLS". My problem with this is that I've been ...
1
vote
0answers
17 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
1answer
38 views

LASSO regression when model is known

I am very new to regression as I have been reading "The Elements of Statistical Learning: Data Mining, Inference, and Prediction" by Hastie et al. on Standford's website this weekend. My goal is to ...
2
votes
1answer
50 views

R LASSO always include some coefficient and question about data partition

I have limited statistic knowledge but I am trying to conduct logistic regression by using a data with 300+ predictors. So I decided to use glmnet and LASSO. Below please see my code: ...
-2
votes
1answer
29 views

Are LASSO coefficients raw or standardized? [on hold]

I'm doing binary classification and ran LASSO to try and do feature selection to reduce the parameters in the model. I have the coefficients from glmnet at the ...
0
votes
1answer
44 views

Optimization with both L1 and L2 regularization

After doing some research I suppose the hard part is that, L2 regularized problem is often solved by gradient descent, while L1 regularized problem is often solved by coordinate descent. But which ...
1
vote
2answers
36 views

R - glasso very slow for high feature space

all, I'm doing a graphical lasso in order to approximate the inverse of the covariance matrix of a 1200 (p-features) by 100 or so (n observations) data matrix. Basically, I'm inverting a 1200 x 1200 ...
3
votes
2answers
64 views

Cross validated penalized logistic regression - one standard deviation rule

I am new to this topic and would like to understand it better. I want to build a binary classifier based on penalized logistic regression. I have 10 features and 23 observations: 16 from class "0" and ...
0
votes
0answers
9 views

How to use adaptive lasso to do subset selection with longitudinal data?

How to use adaptive lasso to do subset selection with longitudinal data? Use R packages Adaptive lasso in R In the website above introduces the R code,but I don't know how to use it with longitudinal ...
0
votes
1answer
36 views

R logistic regression optimal cut point

I am working on a dataset that has 300+ predictors and the dependent variables is very imbalanced (99:1). I need to have a prediction accuracy to show to my client.Here is my analytical process. ...
1
vote
0answers
36 views

How to imagine (visualize) the difference between LARS and Lasso

I'm reading the LARS paper. It turns out the solution path of LARS is quite similar with Lasso, and that paper has an explanation in section 3.1. An important fact ...
1
vote
3answers
96 views

Lasso Regression for predicting Continuous Variable + Variable Selection?

I'm attempting to predict vegetation productivity based on climatic and land use variables (the latter are categorical). I found that there is a multicollinearity problem between the predictors ...
1
vote
0answers
40 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor ...
1
vote
1answer
34 views

Why do I get worse regression metrics when I add more instances to the problem?

I find this counter-intuitive. First I chose randomly 7000 instances and my model explains 55% of the variance. Then I train with the whole dataset (43000) and I get negative $R^2$. How is this even ...
10
votes
0answers
116 views

If the LASSO is equivalent to linear regression with a Laplace prior how can there be mass on sets with components at zero?

We are all familiar with the notion, well documented in the literature, that LASSO optimization (for sake of simplicity confine attention here to the case of linear regression) $$ {\rm loss} = || y - ...
1
vote
0answers
33 views

Meaning and significance testing of coefficients in lasso/ridge regression

Can somebody explain the importance of significance testing in ridge/lasso regression? Is it necessary to do it? And, how can we interpret the coefficients of ridge regression which are penalized?
0
votes
1answer
21 views

Unit tests for lasso?

I have my own implementation for non-negative lasso, but not sure if it is right. Any idea where can I find unit tests to verify my code?
0
votes
0answers
10 views

Lasso - standardize y or only x? [duplicate]

In lasso, I am standardizing the input, x. What do I need to do with the output, y? Are any transformations necessary? Only centering? Both centering and scaling? This question is not a duplicate ...
0
votes
1answer
33 views

Lasso variable selection through correlation?

In lasso, we first standardize all variables to mean=0, var=1. As such, a beta is simply correlation, right? We want to keep the sum of absolute values of betas below certain value, so what do we do: ...
2
votes
1answer
64 views

Elastic net produces complex output with too many non-zero coefficients

I have run 3-fold cross-validation for elastic net using elasticnet R function on ~200 observations and using 80 variables (and there will be some more). Both ...
0
votes
0answers
27 views

Adaptive Group Lasso

Is there any code/algorithm (preferrably for MatLab, but R is fine) where I can easily implement the Adapative Group Lasso as in here. I have found algorithms to implement the group lasso with a ...
0
votes
0answers
6 views

How do I compare the influence of variables against each other?

I've made 22 species specific multiple linear regression models using LASSO and would like to see which variables have the greatest impacts among the models. I'd like to use the parameter values to ...
1
vote
1answer
28 views

How does glmnet() handle with both penalized and unpenalized covariates?

Is it possible to do a lasso model with both penalized and un-penalized covariates? That is, I want to do an estimate with Y ~ gamma * X + beta * Z, where X is a ...
0
votes
0answers
7 views

How to see the adjR-square in Lasso Regression?

After doing lasso, the final parameters are only 6, but I have 200 covariates originally, is it too literally? And how to see the correspond adj R-square in Lasso Regression? By the way, I also tried ...
2
votes
0answers
60 views

Mixed effects Lasso model setup in R, for high dimensional data

My goal is to model the relationship between RETURN and SCORE from my survey dataset with the following structure: RETURN (numeric continuous) = company share price performance SCORE (numeric ...
0
votes
0answers
66 views

Help with modeling an insurance data set with LASSO regression in R

I'm hoping a few folks will help me conceptually with building a model based on an insurance dataset. I'm using LASSO for its feature selection, and I'm using R - probably either the glmnet or ...
2
votes
0answers
50 views

Interpretation of elastic net coefficients under multicollinearity

I am studying the elastic net regression and in some material I read, it was mentioned that the method will choose a group of regressors that are correlated while LASSO can pick one among the ...
7
votes
2answers
369 views

What is elastic net regularization, and how does it solve the drawbacks of Ridge (L2) and Lasso (L1)?

Is elastic net regularization always preferred to Lasso & Ridge since it seems to solve the drawbacks of these methods? What is the intuition and what is the math behind elastic net?
9
votes
2answers
504 views

When will L1 regularization work better than L2 and vice versa?

Note: I know that L1 has feature selection property. I am trying to understand which one to choose when feature selection is completely irrelevant. How to decide which regularization (L1 or L2) to ...
2
votes
0answers
10 views

How to model the discrepancy between test result and prediction code, and find the main sources of discrepancy

I have data about some tested machines, stored as a database with $\approx$ 300 observations, 40 predictors, 2 outcomes. The two outcomes (responses) are the result of the machine test for that ...
2
votes
0answers
20 views

Intepreting significance values for the LASSO regression with covTest in R

The covTest package in R gives significance values for a LASSO regression. How should the results be interpreted? I get negative predictor numbers and NAs for the p-values. What do these mean? More ...
0
votes
1answer
41 views

lasso - how to evaluate results

I'm working on lasso as an alternative to step-wise forward/backward regression using the lars package in R. I normalized my variables, calculated the ...
8
votes
2answers
180 views

Why is Lasso penalty equivalent to the double exponential (Laplace) prior?

I have read in a number of references that the Lasso estimate for the regression parameter vector $B$ is equivalent to the posterior mode of $B$ in which the prior distribution for each $B_i$ is a ...
2
votes
1answer
91 views

Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
0
votes
0answers
19 views

Feature selection for linear regression using bootstrapped RMSE as criteria

I'm trying to build robust linear regression model (lmrob from robustbase) using several (< 15) features. I know that traditional stepwise algorithms aren't the best alternative since they are ...
0
votes
1answer
48 views

True Test Error for LASSO

I have a data set which is split into a training set and a test set. Half is training and half is test. I apply OLS using lm in R, 10 fold Cross Validated LASSO and 10 fold Cross Validated RIDGE using ...
2
votes
0answers
31 views

Conditional Density for Sigma (Bayesian Lasso)

I found that in Bayesian Lasso commonly $\beta \sim N(0,\sigma^2*diag(\tau))$ and $\sigma,\tau \sim \pi(\sigma,\tau)$ is used. Whereas $\pi(\cdot)$ is a product of Laplace distributions. Is it ...
4
votes
1answer
111 views

Scikit's prediction for linear model

Looking at this example for the Lasso method: ...
0
votes
1answer
39 views

GLMNET small deviance explained; reuse of selected predictors in other model

I am trying to run glmnet for logistic regression (I have some continuos predictors which I have scaled with scale() and some categorical which I turned to dummy predictors, 27 predictors, 800 ...
25
votes
2answers
1k views

Why does shrinkage work?

In order to solve problems of model selection, a number of methods (LASSO, ridge regression, etc.) will shrink the coefficients of predictor variables towards zero. I am looking for an intuitive ...
4
votes
1answer
169 views

Why can't ridge regression provide better interpretability than LASSO?

I already have an idea about pros and cons of ridge regression and the LASSO. For the LASSO, L1 penalty term will yield a sparse coefficient vector, which can be viewed as a feature selection method. ...
1
vote
1answer
63 views

Why are all Lasso coefficients in model 0.0?

I'm using from sklearn.linear_model import Lasso in Python 2.7.6 I wrote a script that I've used for doing a Lasso regression for my Features (X) and my Targets ...
0
votes
1answer
44 views

Ordinary linear regression vs penalized regression: predictive MSE

Does predicting on a test set with an ordinary linear regression model result in a smaller predictive MSE compared to a penalized regression model (LASSO or ridge)?
1
vote
0answers
43 views

Subset selection, shrinkage and dimensionality reduction in regression analysis

I am currently reading An Introduction to Statistical Learning: With Applications in R by Robert Tibshirani and Trevor Hastie. I am confused about various regularization methods for linear regression, ...
0
votes
0answers
46 views

Generating Lasso Path for Feature Selection

I am building a Logistic regression model and exploring LASSO for feature selection. I generate the lasso path using the following code: ...
0
votes
0answers
30 views

Why lasso regression is equivalent to finding the least loss function such that $\sum_{j=1}^{p} |\beta_j| \leq \lambda$? [duplicate]

Why is minimizing $\frac{1}{2N} \sum_{i=1}^{N} (y_i - \beta_0 - x_i^T \beta)^2 + \lambda \sum_{j=1}^{p} |\beta_j|$ equivalent to finding the minimum of $\frac{1}{2N} \sum_{i=1}^{N} (y_i - \beta_0 - ...
1
vote
1answer
70 views

Deal with NA's in power transformed data

I'm running a LASSO regression following this guide. I pre - processed my dependent variable using a simple power transformation to obtain a standard normal distribution. Unfortunately, this means I ...
0
votes
1answer
89 views

Standard logistic regression post-Lasso

The situation I'm interested in is logistic regression for a binary response variable with lots of predictors (500 to 1000), lots of which are correlated. I would like to use a logistic LASSO approach ...