Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

0
votes
1answer
20 views

How to select the important feasures from lasso model?

I used different initial value and applied the interactive method to fit the lasso model (1000 sample size, 5000 features), but the non-zero coefficients are different every time I changed the initial ...
0
votes
0answers
9 views

How to solve an adaptive lasso model?

Assuming we are working with a linear regression model, lasso penalization solves: \begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^p \left\vert \beta_j\...
0
votes
0answers
5 views

The consequences of ignoring autocorrelation of errors for the LASSO estimator?

In ordinary linear regression, Y = X$\beta$ + $\epsilon$, if the error is autocorrelated, then the assumptions under the Gauss-Markov theorem are violated. For example, autocorrelation violates the ...
1
vote
0answers
9 views

Is it a good idea to do Cross-Validation for LASSO with a small sample size?

I have a dataset consist of 40 rows and 15 terms as variables. I need to develope a "prediction model" based on LASSO classification. Thus, I want know the best significant terms with their ...
0
votes
0answers
19 views

Is adaptive lasso still unbiased for glm(such as Logistics)?

I'm doing something about penalized Logistics regression with adaptive LASSO recently. But I found that the coefficients from Logisitcs+adaptive LASSO is quite different from the normal Logistics ...
0
votes
0answers
10 views

How does Cross Validation work in Matlab

I am doing some research on Lasso classification method. I have a 40x15 dataset and I want to develop a binomial equation without dividing data into train and test set (because of small sample size). ...
0
votes
1answer
24 views

Beta Regression Model Selection with CV Lasso in R

Is there a package that will do a cross-validatation with regularization for beta regression in R? I'm looking for an equivalent of glmnet for the betareg package.
0
votes
0answers
13 views

How exactly Matlab performs the Lasso classification?

I am doing some research on Lasso classification method. I have a 40x15 dataset and I want to develop a binomial equation without dividing data into train and test set (because of small sample size). ...
0
votes
0answers
23 views

LassoCV regression on price returns doesn' t work

I'm trying to use LassoCV to get a linear model for the log return of an asset price. So what I am doing is: Download historical prices for near 61 assets Calculates the log return and difference of ...
0
votes
0answers
21 views

Optimizer for $L_1$ (LASSO) penalty: L-BFGS, COBYLA, OWL-QN or other?

I am sure there are other sources, but reading the following vignette of R package lbfgs, it is claimed that L-BFGS optimization algorithm is not suited for an ...
0
votes
1answer
24 views

Why Lasso classification results changes in Matlab?

I am doing lasso classification using Lassoglm command in Matlab. I have a problem and that is, every time I run the program for my dataset I get different variables to have non-zero coefficients. I ...
1
vote
0answers
16 views

lambda value threshold beyond which shrinkage penalty in glm is high and coefficients approach zero

Conceptually, for both LASSO and ridge regression methods as lambda becomes "very large", the penalty impact grows and the coefficients approach zero. However, a) is there a particular threshold for ...
0
votes
0answers
7 views

Trouble with implementing Lasso Regression using Coordinate Descent

https://github.com/thefr33radical/projects/blob/master/research/LassoRegression_p.ipynb I am having trouble understanding Coordinate Descent used on Lasso. Please could someone explain in a simple ...
0
votes
0answers
8 views

Do I need to do glmnet after doing a cv.glmnet?

I'm studying now about the model selection from the ISLR book. I'm don't understand about whether should I do glmnet() after I do ...
0
votes
3answers
39 views

Lasso Regression as Variable Selection

Suppose we are initially given $p$ predictor variables. In lasso regression, we want to find estimates of the coefficients $\beta_1, \dots, \beta_p$ that minimize $\text{RSS}+ \lambda \sum_{j=1}^{p} |\...
1
vote
0answers
28 views

Is group lasso equivalent to ridge regression when there is 1 group

On Wikipedia, it says that: "while if there is only a single group, it reduces to ridge regression" (https://en.wikipedia.org/wiki/Lasso_(statistics)#Group_lasso). However in group lasso we have norm ...
1
vote
1answer
24 views

glmnet package: “mgaussian” vs “gaussian” for $\alpha = 0$

In multiresponse Gaussian family the objective function when $\alpha = 0$: \begin{align} \frac{1}{2n}||Y-XB||_F^2 + \frac{\lambda}{2}||B||_F^2. \end{align} This can also mathematically solved as \...
0
votes
2answers
26 views

Relaxed Lasso Logistic Regression: Estimating second penalty parameter

I'm trying a relaxed lasso logistic regression by first using sklearn's cross validation to find an optimal penalty parameter (C = 1/lambda). Then, I use that parameter to fit statsmodel's logit model ...
0
votes
0answers
18 views

Ridge, Lasso or Elastic nets used in Accounting Research

I am trying to come up with ideas for my master's thesis and was wondering why literature on the above mentioned regression methods within Accounting Research is non-existent? I felt like the ...
0
votes
0answers
17 views

Why does Group Lasso use L2 norm for individual group penalties?

In group lasso $$\min_{\beta}\left\{\frac{1}{2} \left\lVert{y}-\sum_{l=1}^mX^{(l)}{\beta^{(l)}}\right\rVert_2^2 +\lambda\sum_{l=1}^m\sqrt{p_l}\left\lVert{\beta^{(l)}}\right\rVert_q\right\}$$ the ...
2
votes
0answers
15 views

LASSO Cox Model after multiple imputation

I want to develop a predictive survival model on a data set with about 8000 subjects and 38 covariates. About 4% of subjects had the event of interest. There are 21 variables with missing values, ...
0
votes
0answers
37 views

Tuning glmnet hyperparameters in MLR

I want to estimate LASSO using glmnet in MLR with spatial cross-validation to tune lambda. Questions: In makeParamSet, do I specify ...
0
votes
0answers
14 views

Cross-validation with one-standard-rule for Elastic Net

I would like to use one-standard -deviation rule for my hyper-parameter selection for elastic net. I understand how to do it for ridge or lasso, but when it comes to two regularization parameters I am ...
0
votes
1answer
65 views

Bad Linear regression results

I have a dataset, and i have to predict the flow of users at a certain city given some information like the day of the week, the month, the distance of the city of origin ecc.. First i decided to ...
0
votes
0answers
14 views

Probability that some coefficients will be zero with lasso regression

Maybe it is duplicate, but I don't know how to ask a right question. Suppose I fixed some lambda, for instance lambda = 1. I draw the usual "square" |x| + |y| = 1. Now I want to find probability, ...
0
votes
1answer
49 views

Lasso cox regression with bootstrap

I'm looking at building a nomogram for cancer prognosis based on 20 variables. This will be derived from a cox ph model. In the past I used poor methodology including dichotomization and stepwise ...
2
votes
0answers
35 views

LASSO solution when $p \gg n$

We know that when $p \gg n$ we don't have a unique least squares solution. But, in the same scenario, does LASSO have a solution? LASSO uses the least squares term as part of its cost function (i.e.,...
5
votes
1answer
70 views

Literature on $\ell_q$ LASSO, $q < 1$

I am not sure how is $\ell_q$-LASSO called, but here I am talking about LASSO regression, with $\| \beta \|_{\ell_q}$ regularization, $q< 1$. In popular literature, such as Elements of Statistical ...
5
votes
1answer
88 views

Regression - variance of predictions much lower than variance of target

I am using non-negative lasso(sklearn) on a dataset with 1.5MM data points and 120 features. It is a low R2 environment (working with noisy financial data), so $R^2$ is about 10%. What I am more ...
0
votes
0answers
21 views

Equivalence between Stepwise Regression and Lasso

A while ago I had learned of a theoretical result that suggested there was a correspondence with Forward/Backward/Stepwise regression and LASSO/RIDGE regression in terms of the coefficient of ...
1
vote
0answers
53 views

LASSO regression: which method is better for selecting $\lambda$ in this case?

I am currently working on a method for adaptive knot placement in Spline regression. Following Osborne et.al. (1998), Yuan et.al. (2014) I am interested in using LASSO regression to select a subset of ...
0
votes
1answer
62 views

Is there an R function that performs LASSO regression on multiple imputed datasets and pools results together?

I have a dataset with 283 observation of 60 variables. My outcome variable is dichotomous (Diagnosis) and can be either of two diseases. I am comparing two types of diseases that often show much ...
2
votes
0answers
30 views

Incorrect computation in Knight and Fu (2000)?

I'm currently reading Knight and Fu's 2000 paper on the asymptotics of "Bridge" estimators with a particular focus on LASSO as a special case. In the proof of theorem 2, they make the claim that under ...
1
vote
1answer
51 views

Is there any two-stage procedure for elastic net as LASSO?

I read this post Why use Lasso estimates over OLS estimates on the Lasso-identified subset of variables? . It says the LASSO shrinkage causes the estimates of the non-zero coefficients to be biased ...
1
vote
0answers
14 views

A generalized LASSO constraint

I want to use LASSO in R but shrinking towards some fixed vector $A$, instead of shrinking towards 0. The desired L1 constraint, given coefficient vector $\beta$, is $||\beta-A||_{1} \leq k$, rather ...
0
votes
1answer
17 views

Feature selection by lasso in two samples compared to one joint sample

Let's say you have two sets of features $X_1$ and $X_2$ together with a response variable $Y$. I wonder whether the two following procedures are identical asymptotically (or in finite samples) in ...
0
votes
0answers
27 views

Fitting 2-dimensional data with LASSO?

I have a problem where I need to fit two-dimensional data. The matrix is of size 10x1000, where the rows correspond to discretely measured time points, and the columns correspond to measured spectra ...
0
votes
0answers
15 views

Sparse solutions: linear systems vs logistic regression

It is known in the field of compressed sensing/sparse approximations that if $$Ax = b$$ has sparse solution (with $s$ nonzeros), then there is a condition which states that it is unique, if $$s \leq 0....
0
votes
0answers
29 views

How is the generalization of LASSO called?

I know that ridge regression is a special case of Tykhonv regularization. In fact with Tykhonov one tries to minimize: $|| Ax - b ||^2 +|| \Gamma x ||^2$ If $\Gamma$ is the identity matrix scaled by ...
1
vote
1answer
34 views

State space with lasso

Is it possible to incorporate lasso variable selection in the high dimensional state space model. If yes, is there any code or package available in R
2
votes
0answers
59 views

Why not use Ridge after Lasso vs relaxed Lasso

Has anyone ever applied a ridge regression on a model subset selected from a cross validated lasso? In other words, take a data set with p features and run lasso, grid searched to find optimal ...
2
votes
1answer
65 views

Can I use PCA after lasso variable selection?

I have a data regarding life satisfaction, of more than 2000 observations and 265 variables (most are categorical variables). I want to build a model, estimating the effects of society problems on the ...
1
vote
1answer
45 views

why does lasso select at most n predictors?

From the seminal paper on elastic net regularization from Zou and Hastie 2005, I read ...
0
votes
1answer
32 views

Application of LASSO , Ridge, PLS in feature selection of spectral data

The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based) But none of the features are removed ( no ...
0
votes
1answer
28 views

Logistic regression with coefficients penalized to other numbers

When you penalize logistic regression using l1, l2 or both penalizations, the coefficients are penalized towards 0. I would like to do the same thing but penalizing the coefficients towards other ...
0
votes
0answers
25 views

Additive Gaussian Processes with Penalized Likelihood

I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...
0
votes
0answers
30 views

Trouble understanding L1 and L2 cost function [duplicate]

When reading the Sklearn User Guide, one might see the following statement about Logistic Regression As an optimization problem, binary class L2 penalized logistic regression minimizes the ...
0
votes
0answers
54 views

Can you do LASSO with panel data that has cross-correlated errors?

suppose that there are observations of stocks return for firm i in 1..N date of observations t in 1..T The error structure is such that cov(epsilon_it,epsilon_jt) are far from zero. An important ...
3
votes
0answers
15 views

How sensitive is $L_p$ regression to initialisation?

Consider that I wish to solve a linear regression in the $L_p$ framework That is, the optimisation problem that I wish to solve is $$ \mathbf{w} = \text{argmin}_{\mathbf{w}} ||\mathbf{w}^T\mathbf{x} -...
2
votes
1answer
60 views

Consistency of Adaptive LASSO

I'm reading the paper on Adaptive LASSO estimator (Zou, 2006). In one of the presented numerical simulation examples (Model 0 (Inconsistent lasso path), page 6 (1423)) they claim the following: To ...