Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

0
votes
0answers
4 views

Lasso regression with lasso2 (l1ce) vs glmnet

I'm struggling to get the same results from a lasso regression when using glmnet as when using l1ce from the lasso2 package. I've set a specific tuning parameter value for both, and tried to set all ...
0
votes
0answers
14 views

Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
2
votes
1answer
35 views

Alternatives to Pre-Scaling Predictors in Lasso/Ridge Regression?

In lasso/ridge regression it's often recommended to scale predictors $X$ before estimation so that the coefficient estimates $\hat{\beta}$ will be invariant to the scale of predictors $X$. Q: Is ...
0
votes
2answers
62 views

Choosing model for more predictors than observations

I'm working with a data consisting of 1000 observations of 2000 predictors and one variable we wish to predict. There are couple of problems I can't get around. I am aware that such setting has been ...
0
votes
0answers
17 views

LOOCV in Caret works with Glmnet and not ElasticNet

I'm a phd student learning about different machine learning and cv methods so i apologize if this is a silly question. I have a decent understanding of lasso and am using the ...
1
vote
0answers
24 views

LASSO method. Intuitively how does it select variables? [duplicate]

Intuitively how does the LASSO method select its variables? Is it based on standard econometrics?
0
votes
0answers
12 views

Is lasso appropriate on full binary dataset? (R) [duplicate]

Y is the dependent variable with the outcome 0 and 1, and so are X1...X140. As far as I know, I can't use the simple lasso regression in order to look at which variables are shrinked down, since we ...
0
votes
0answers
22 views

Group lasso coordinate descent algorithm

The group lasso problem is $$ \min_{\boldsymbol{\beta}}\left\Vert \mathbf{y}-\sum_{j=1}^{m}\mathbf{X}_{j}\boldsymbol{\beta}_{j}\right\Vert _{2}^{2}+\sum_{j=1}^{m}\lambda_{j}\left\Vert \boldsymbol{\...
0
votes
1answer
31 views

Why does LASSO ignore a predictor that has predicting power and NOT correlated with other predictors?

I have a linear regression problem for my car fleet data, where $y$ is the change in rental price and $X$ is a design matrix with around 30 columns (predictors). Most of the predictors are continuous ...
0
votes
0answers
12 views

3rd interactions using LASSO

This may be an easy/wrong question: I am trying to find third order interactions and which data to keep in my model: I am using LASSO and the glmnet package in R. I have multiple variables, E1,E2,E3,...
5
votes
2answers
215 views

One-to-one correspondence between penalty parameters of equivalent formulations of penalised regression methods

Ridge, LASSO and Elastic Net are three very popular methods of penalised regressions. All of these have more than one formulations. For example, two formulations for Ridge are: minimise $\lVert Y - X ...
0
votes
0answers
29 views

Will LASSO choose variables that are highly correlated with the outcome variable?

Suppose we have access to an outcome variable $Y_i$ and a $p$-dimensional vector $X_i$ for $i=1,\ldots,N$. We run a LASSO regression of $Y$ on $X$ for every penalty/shrinkage parameter $\lambda$ in an ...
0
votes
0answers
11 views

How to choose $\lambda$ is compressive sensing

I am trying to reconstruct a signal using basis pursuit denoising of the compressive sensing framework (which is basically lasso), $min_{x} \frac{1}{2} || y − Ax||_2^2 + \lambda ||x||_1$. Here x is ...
2
votes
0answers
16 views

Lasso acts differently for a large (1mi obs) sample? [closed]

I am fitting Lasso using the glmnet package in R. The data contains 1 million observations and 1500 predictors. We have a survival outcome (time to death) ...
0
votes
0answers
28 views

How to choose the optimum value of regularization parameter $\lambda $ to generate saprsity by using lasso-cross-validation?

I am working on an underwater sonar experiment and I have the transmitted signal sample data and received signal samples data with me. By using this data I am forming the linear model $x=\psi\theta$....
0
votes
1answer
28 views

Writing by hand first steps in Least Angle Regression (LARS)

How do we write the first steps of Least Angle Regression ? What is the rationale behind this method ? What limitations of other methods is it overcoming ? Why is it called Least Angle Regression ?
0
votes
1answer
45 views

How do I make predictions from Lasso coefficients?

I am struggling to understand the implementation of lasso regularization (LassoCV in sklearn) and feature selection. First, I used cross-validation to determine value of alpha that minimizes the MSE. ...
1
vote
1answer
34 views

Interpretation of cross validation plot for Lasso regression

I am trying to understand the plot below generated in R (using the function cv.glmnet) which illustrates the cross validation process for picking the value of lambda in lasso regression. What are the ...
2
votes
1answer
64 views

Why is lasso more robust to outliers compared to ridge?

In my attempt to reason about it intuitively I am concluding that ridge might be more robust to outliers. Following is my intuitive/lose reasoning : If there is an outlier then to match my ...
3
votes
0answers
20 views

Before using CV-selected Regression model for Inference, shouldn't model performance be evaluated on unused test set?

I just came across a biokinesiology paper that used some Machine Learning methods, but I think there is a flaw in their methodology. The authors had data on stroke patients and used Lasso regression ...
0
votes
0answers
44 views

LASSO Regression using Panel Data

I have panel data for 3 countries, ranging over 3 years. The dataset is called CarProduction ...
0
votes
0answers
18 views

Performing lasso - Differential Expression of genes

I have a data set of dimensions 19000 x 288, where there are 19000 transcripts (variables) and 288 observations. The observations correspond to 32 individuals from two countries (16 from Kenya, 16 ...
0
votes
0answers
26 views

Regularization techniques (ridge, lasso or elastic-net regression) for meta-analysis

I was perusing the StatQuest YouTube channel, and serendipitously listened to a series of videos on regularization techniques (ridge, lasso or elastic-net regression). It came to my mind that such ...
0
votes
0answers
15 views

scaling maximum likelihood function with the amount of observations

I am a bit confused about the formulation of the maximum liklihood equation for logistic regression for ridge regression (and similar for lasso regression). Where andrew Ng (coursera course) states ...
1
vote
2answers
72 views

How to interpret / metric Lasso regression coefficients

Edited Question, since it was a duplicate I used Matlab to make a lasso model for my data that has 41 predictors and 1 response variable, and perhaps I used more variables that I need too or maybe ...
2
votes
0answers
20 views

High dimensional linear regression inference

I am reading through high-dimensional literature currently but I got confused. Especially about statistical inference with LASSO, anyone can clarify me the main difference among Van De Geer(2014)ON ...
0
votes
0answers
18 views

Use MSE in cv.glmnet for Poisson models?

I want to compare different methods (like Poisson regression using Lasso, a convolutional NN, etc.) in terms of prediction error. As error measures I chose the MSE, the MdAPE (median absolute ...
2
votes
1answer
296 views

How to obtain Confidence Intervals for a LASSO regression?

I'm very new from R. I have this code for a LASSO regression: ...
1
vote
1answer
23 views

How does feature selection work for non linear models?

A model like a neural network or an SVM is called for only if the interactions between the features and the target is non-linear, otherwise we're better off using linear or logistic regression. But ...
0
votes
0answers
27 views

GLMNET LASSO - interpretation problem

We have a sample artificial dataset. The response variable y is binomial categorical: ...
0
votes
0answers
26 views

Lasso Logistic Regression in the presence of Class Imbalance

Since class imbalance only affects the estimate of the intercept in vanilla logistic regression, the orientation of the optimal separating hyperplane remains unchanged. However in $L_1$-regularized ...
2
votes
1answer
50 views

Interpreting coefficients and understanding logistic lasso

In lasso regression, increasing the regularisation strength/shrinkage penalty eventually forces all of the regression coefficients to zero. In this instance the regression is logistic. The plot shows ...
10
votes
2answers
443 views

Showing the Equivalence Between the $ {L}_{2} $ Norm Regularized Regression and $ {L}_{2} $ Norm Constrained Regression Using KKT

According to the references Book 1, Book 2 and paper. It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...
1
vote
3answers
59 views

LASSO or random forest (RF) to use for variable selection when having highly correlated features in a relatively small dataset with many features?

I have a cross sectional data-set with around 1000 features and 5000 observations. There are many features (no categorical features) which are highly correlated (higher than 0.85). I want to decrease ...
1
vote
1answer
67 views

Lasso for time series - Independence assumptions violated?

I know it is not uncommon to use LASSO for time series. But as LASSO is actually only linear regression or a GLM with a constraint, what about the assumption of independent observations these models ...
0
votes
2answers
64 views

Feature selection using PCA for linear regression

I am using PCA to the training data set to do feature selection before applying linear regression to build a classifier model. In this scenario, would it be useful to use ridge regression to ensure ...
0
votes
1answer
25 views

CrossValidation in ElasticNet Lambda Parameter

Generally, 10-fold CV is used to find the best $\lambda$ (the shrinkage parameter, not the trade-off between L1 and L2 norm $\alpha$ - see https://stats.stackexchange.com/a/64278/185237) in an elastic ...
2
votes
0answers
94 views

Implementing Lasso Regression in Numpy

I'm doing a little self study project, and am trying to implement OLS, Ridge, and Lasso regression from scratch using just Numpy, and am having problems getting this to work with Lasso regression. ...
0
votes
0answers
28 views

Using Quadratic Programming to solve Lasso and Ridge regression models?

I'm trying to build linear, ridge and lasso regression models for at set of data (40 obs., 4 features, 1 response). I'm building the models using the sklearn package for Python and I can easily find ...
4
votes
1answer
87 views

How to solve an adaptive lasso model?

Assuming we are working with a linear regression model, lasso penalization solves: \begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^p \left\vert \beta_j\...
0
votes
0answers
18 views

The consequences of ignoring autocorrelation of errors for the LASSO estimator?

In ordinary linear regression, Y = X$\beta$ + $\epsilon$, if the error is autocorrelated, then the assumptions under the Gauss-Markov theorem are violated. For example, autocorrelation violates the ...
1
vote
1answer
19 views

Is it a good idea to do Cross-Validation for LASSO with a small sample size?

I have a dataset consist of 40 rows and 15 terms as variables. I need to develope a "prediction model" based on LASSO classification. Thus, I want know the best significant terms with their ...
0
votes
0answers
38 views

Is adaptive lasso still unbiased for glm(such as Logistics)?

I'm doing something about penalized Logistics regression with adaptive LASSO recently. But I found that the coefficients from Logisitcs+adaptive LASSO is quite different from the normal Logistics ...
0
votes
0answers
22 views

How does Cross Validation work in Matlab

I am doing some research on Lasso classification method. I have a 40x15 dataset and I want to develop a binomial equation without dividing data into train and test set (because of small sample size). ...
0
votes
1answer
60 views

Beta Regression Model Selection with CV Lasso in R

Is there a package that will do a cross-validatation with regularization for beta regression in R? I'm looking for an equivalent of glmnet for the betareg package.
0
votes
0answers
20 views

How exactly Matlab performs the Lasso classification?

I am doing some research on Lasso classification method. I have a 40x15 dataset and I want to develop a binomial equation without dividing data into train and test set (because of small sample size). ...
0
votes
0answers
26 views

LassoCV regression on price returns doesn' t work

I'm trying to use LassoCV to get a linear model for the log return of an asset price. So what I am doing is: Download historical prices for near 61 assets Calculates the log return and difference of ...
0
votes
1answer
57 views

Why Lasso classification results changes in Matlab?

I am doing lasso classification using Lassoglm command in Matlab. I have a problem and that is, every time I run the program for my dataset I get different variables to have non-zero coefficients. I ...
1
vote
0answers
26 views

lambda value threshold beyond which shrinkage penalty in glm is high and coefficients approach zero

Conceptually, for both LASSO and ridge regression methods as lambda becomes "very large", the penalty impact grows and the coefficients approach zero. However, a) is there a particular threshold for ...
0
votes
0answers
31 views

Trouble with implementing Lasso Regression using Coordinate Descent

https://github.com/thefr33radical/projects/blob/master/research/LassoRegression_p.ipynb I am having trouble understanding Coordinate Descent used on Lasso. Please could someone explain in a simple ...