Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

Filter by
Sorted by
Tagged with
0
votes
0answers
37 views

Can data ever be too high dimensional for the Lasso?

I'm trying to implement Lasso on high dimensional textual data. Format of Data: p ~= 45,000, n~=4,000 When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
0
votes
0answers
16 views

Adaptive Lasso:msgps, glmnet, or glmaag?

To perform Adaptive lasso, which is the best package: msgps, or glmnet or glmaag? What's the difference among the three packages: algorithm, results?
0
votes
1answer
28 views

implement lasso Regression in GAMs in R

Hello I search a way to select variables in a gam function R by using Lasso Regression. I already fitted a model gam(y~x) with x ...
0
votes
0answers
15 views

Creating a risk score from Cox Regression

I have two datasets with palliative cancer patients including 106 and 60 patients, respectively. I have biomarkers of inflammation and coagulation, as well as clinical characteristics for all patients....
0
votes
0answers
20 views

Mathematical proof of how L1 and L2 regularization work [duplicate]

How do you mathematically prove that L1 regularization makes weights sparse but L2 regularization does not?
0
votes
1answer
36 views

Cox regression with lasso regression

Is it possible to perform lasso regression (glmnet with "cox") for variable selection and then conduct Cox regression using ...
1
vote
2answers
100 views

Minimum and maximum regularization in L0 (pseudo)norm penalized regression

L0-pseudonorm penalized least squares regression (aka best subset regression) solves $\widehat{\beta}(\lambda)$ as $$\min_\beta \frac{1}{2}||y-X\beta||_2^2 +\lambda||\beta||_0.$$ where $||\beta||_0$ ...
0
votes
0answers
15 views

How can i compare LASSO regression with output of STEPWISE logistic regression? [closed]

What kind of evaluation should i use to comapre both these type of LR. Can i compare the MSE of both. If yes, then how can i do that with glm & glmnet packages. Other than that i just have ...
0
votes
0answers
23 views

Are there any evaluation tests for LASSO logistic regression?

As we have Hosmer lemshow test,McFadden R2, Concordance tests for Logistic regression. Do we have any such tests for evaluating the performance of LASSO?
0
votes
0answers
20 views

How to interpret scores from regression models? [duplicate]

Using Kaggle House Price dataset, I splitted my data into training set and test set and am running validation on it. ...
0
votes
0answers
8 views

Interpreation of beats in lasso regression

I have a dataset with lot´s of variables and I want to try to use the lasso regression. My question is how to interpret the results. As an example, I use the Hitters dataset, in which the individual ...
2
votes
1answer
38 views

Can Park & Casella's Bayesian LASSO be applied to generalized linear models?

In Park & Casella's Bayesian LASSO model the LASSO is estimated through a scale mixture of normals: ...
0
votes
0answers
8 views

Does elastic net with Arima errors make sense?

I know of regression with arima errors, but can one also do elastic net regression with arima errors? I ask because I read somewhere that the residuals for elastic net are not really valid since ...
0
votes
0answers
6 views

What is the meaning of assuming a special prior on regularization method

I have heard/read that L1 regularization assumes Laplacian prior, however L2 regularization assumes Gaussian prior. But what exactly "assume" mean here? How does it work? How do each of these ...
1
vote
0answers
5 views

Does the initial distribution of data have any affect on which regularization parameter can work well?

In scenarios when we want to know why performance of a predicting linear regression model when using L1 regularization has outperformed with the case that we have used L2 regularization, I wonder ...
0
votes
0answers
12 views

Lasso logistic regression with identical features

I am working on a model and accidentally passed in two identical features (with different names) to a logistic regression model with Lasso. Instead of dropping one of them and keeping the other, the ...
0
votes
0answers
13 views

Solving the Binary Logistic Regression with LASSO penalty

The objective function of the Binary logistic regression with the LASSO penalty is given by, $argmin_{\beta_0,\beta}$ { $-{1}/{n}$ $\sum_{i=1}^n (Y_i({\beta_0}+{\beta^T}x_i)-log(1+exp({\beta_0}+{\...
5
votes
2answers
195 views

Understanding Lasso Regression's sparsity geometrically

Whenever someone writes about Lasso and Ridge Regression thy draw this diagram with the circle or with the diamond. In the case of the diamond (Lasso regression) it is then always stated that Lasso ...
0
votes
0answers
9 views

Coordinate Descent for the Binary Logistic Regression

I am studying Binary Logistic Regression (BLR) with the LASSO penalty and am trying to solve my objective function using the coordinate descent as discussed in the paper by https://web.stanford.edu/~...
0
votes
1answer
21 views

Binary Logistic Regression with the LASSO objective function

I am working on my MSc. Statistic which is on the Penalized Logistic Regression with the LASSO penalty. I am trying to understand the difference in two objective functions: argmin {$\frac{1}{n}$ $\...
2
votes
1answer
31 views

Elastic net/LASSO with soft labels

Sometimes you do not have firm Y/N labels, but e.g. 80% probability of Y as a label. E.g. this happens, if you train a model on a small amount of labelled data, predict for a large amount of ...
0
votes
0answers
14 views

Derviation of Lasso dural form problem

On page 13 of https://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-110.html, "For each i, consider each optimization problem" $ \max\limits_{z_i}u_i^Tz_i-m_i log\left(\sum_{j=1}^n \delta_{ij} ...
0
votes
0answers
8 views

Lasso for multi-output regression giving same results for all alpha values

I am using Lasso for multi-output regression. However, whatever value of alpha I am using, it is producing the same mse and R^2 values. Am I doing something wrong? I have tested the code with ...
0
votes
0answers
12 views

Multivariate sparse regression - selective predictors

What do I do to find predictors that are ONLY/mostly related to one of several continuous response variables ? E.g., if you have (continuous) outcomes y1,y2,y3 and predictors a,b,c,... and a is ...
0
votes
0answers
19 views

Elastic net regression with uneven penalties for predictors

For a regression model where you are certain that y that depends on some predictors but are agnostic about whether some other predictors should enter, how should you incorporate this prior information?...
0
votes
1answer
14 views

Why are oracle inequalities called that way?

The oracle property is an asymptotic property of an estimator, and is about variable selection: An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
2
votes
0answers
27 views

Why does lasso return unstable features when using the same data?

I am using scikit-learn to shrink my data set having around 800 features. It is a very noisy data (market and economic data) To my best knowledge, lasso returns same features for the same data set. ...
14
votes
0answers
214 views

Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
0
votes
0answers
16 views

GLMM with Variable Selection and Non-Negativity Constraint

I am trying to run a fairly complex GLMM with random effects and smooths. There are about 10 of these independent variables. There is also another set of 1000 variables. From this set of 1000 ...
0
votes
0answers
22 views

Plot interpretation for underfitting/overfitting - Lass cv.glmnet

I ran a logistic Lasso Regression in R using the cv.glmnet package and get the plot below. I understand that the left dotted line is lambda.min and the right dotted line is lambda.1se. I also see ...
0
votes
1answer
41 views

Why is Lasso and Ridge not giving better results than OLS?

I am trying to find an example in which Lasso and Ridge regression are doing better than simple OLS. I am trying to run the Boston example that appears in the MASS library in R. The dependent ...
0
votes
0answers
26 views

Ridge vs. Lasso vs. Elastic Net [duplicate]

I have a theoretical question. I was reading about ridge regression, lasso and the elastic net, and is very impressed. One thing is not quite clear to me. I would like to know when should I use each ...
6
votes
2answers
261 views

LASSO Regression - p-values and coefficients

I've run a LASSO in R using cv.glmnet. I would like to generate p-values for the coefficients that are selected. I found the boot.lass.proj to produce bootstrapped ...
2
votes
1answer
49 views

Covariance of an estimate from optimization

Consider a standard linear regression model, $\boldsymbol y = X \boldsymbol \beta + \boldsymbol \epsilon$. $\boldsymbol y$ is a vector of $m$ responses, $X$ is a design matrix with $m$ rows and $p$ ...
0
votes
0answers
14 views

What is the scale of pseudo R squared obtained from my Logit model?

I am getting a pseudo R squared in the range of 0.01 - 0.05 when I experiment with various combination of features. I am aware of this post: McFadden's Pseudo-R2 Interpretation says 0.2-0.4 ...
1
vote
0answers
30 views

How does regularized regression overcome the p > n problem?

So, I understand why simple linear or logistic regression will have infinite solutions in this case (good answers here and here). But while LASSO will only select n features, Elastic net does not have ...
0
votes
0answers
13 views

Lasso regression with lasso2 (l1ce) vs glmnet

I'm struggling to get the same results from a lasso regression when using glmnet as when using l1ce from the lasso2 package. I've set a specific tuning parameter value for both, and tried to set all ...
1
vote
0answers
26 views

Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
3
votes
1answer
42 views

Alternatives to Pre-Scaling Predictors in Lasso/Ridge Regression?

In lasso/ridge regression it's often recommended to scale predictors $X$ before estimation so that the coefficient estimates $\hat{\beta}$ will be invariant to the scale of predictors $X$. Q: Is ...
0
votes
2answers
75 views

Choosing model for more predictors than observations

I'm working with a data consisting of 1000 observations of 2000 predictors and one variable we wish to predict. There are couple of problems I can't get around. I am aware that such setting has been ...
0
votes
0answers
27 views

LOOCV in Caret works with Glmnet and not ElasticNet

I'm a phd student learning about different machine learning and cv methods so i apologize if this is a silly question. I have a decent understanding of lasso and am using the ...
1
vote
0answers
31 views

LASSO method. Intuitively how does it select variables? [duplicate]

Intuitively how does the LASSO method select its variables? Is it based on standard econometrics?
0
votes
0answers
13 views

Is lasso appropriate on full binary dataset? (R) [duplicate]

Y is the dependent variable with the outcome 0 and 1, and so are X1...X140. As far as I know, I can't use the simple lasso regression in order to look at which variables are shrinked down, since we ...
0
votes
0answers
28 views

Group lasso coordinate descent algorithm

The group lasso problem is $$ \min_{\boldsymbol{\beta}}\left\Vert \mathbf{y}-\sum_{j=1}^{m}\mathbf{X}_{j}\boldsymbol{\beta}_{j}\right\Vert _{2}^{2}+\sum_{j=1}^{m}\lambda_{j}\left\Vert \boldsymbol{\...
0
votes
1answer
34 views

Why does LASSO ignore a predictor that has predicting power and NOT correlated with other predictors?

I have a linear regression problem for my car fleet data, where $y$ is the change in rental price and $X$ is a design matrix with around 30 columns (predictors). Most of the predictors are continuous ...
0
votes
0answers
14 views

3rd interactions using LASSO

This may be an easy/wrong question: I am trying to find third order interactions and which data to keep in my model: I am using LASSO and the glmnet package in R. I have multiple variables, E1,E2,E3,...
5
votes
2answers
230 views

One-to-one correspondence between penalty parameters of equivalent formulations of penalised regression methods

Ridge, LASSO and Elastic Net are three very popular methods of penalised regressions. All of these have more than one formulations. For example, two formulations for Ridge are: minimise $\lVert Y - X ...
0
votes
0answers
31 views

Will LASSO choose variables that are highly correlated with the outcome variable?

Suppose we have access to an outcome variable $Y_i$ and a $p$-dimensional vector $X_i$ for $i=1,\ldots,N$. We run a LASSO regression of $Y$ on $X$ for every penalty/shrinkage parameter $\lambda$ in an ...
0
votes
0answers
16 views

How to choose $\lambda$ is compressive sensing

I am trying to reconstruct a signal using basis pursuit denoising of the compressive sensing framework (which is basically lasso), $min_{x} \frac{1}{2} || y − Ax||_2^2 + \lambda ||x||_1$. Here x is ...
2
votes
0answers
16 views

Lasso acts differently for a large (1mi obs) sample? [closed]

I am fitting Lasso using the glmnet package in R. The data contains 1 million observations and 1500 predictors. We have a survival outcome (time to death) ...