Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

1
vote
0answers
26 views

Is there any two-stage procedure for elastic net as LASSO?

I read this post Why use Lasso estimates over OLS estimates on the Lasso-identified subset of variables? . It says the LASSO shrinkage causes the estimates of the non-zero coefficients to be biased ...
0
votes
0answers
8 views

A generalized LASSO constraint

I want to use LASSO in R but shrinking towards some fixed vector $A$, instead of shrinking towards 0. The desired L1 constraint, given coefficient vector $\beta$, is $||\beta-A||_{1} \leq k$, rather ...
0
votes
1answer
15 views

Feature selection by lasso in two samples compared to one joint sample

Let's say you have two sets of features $X_1$ and $X_2$ together with a response variable $Y$. I wonder whether the two following procedures are identical asymptotically (or in finite samples) in ...
0
votes
0answers
19 views

Fitting 2-dimensional data with LASSO?

I have a problem where I need to fit two-dimensional data. The matrix is of size 10x1000, where the rows correspond to discretely measured time points, and the columns correspond to measured spectra ...
0
votes
0answers
7 views

Sparse solutions: linear systems vs logistic regression

It is known in the field of compressed sensing/sparse approximations that if $$Ax = b$$ has sparse solution (with $s$ nonzeros), then there is a condition which states that it is unique, if $$s \leq 0....
0
votes
0answers
22 views

How is the generalization of LASSO called?

I know that ridge regression is a special case of Tykhonv regularization. In fact with Tykhonov one tries to minimize: $|| Ax - b ||^2 +|| \Gamma x ||^2$ If $\Gamma$ is the identity matrix scaled by ...
1
vote
1answer
23 views

State space with lasso

Is it possible to incorporate lasso variable selection in the high dimensional state space model. If yes, is there any code or package available in R
2
votes
0answers
37 views

Why not use Ridge after Lasso vs relaxed Lasso

Has anyone ever applied a ridge regression on a model subset selected from a cross validated lasso? In other words, take a data set with p features and run lasso, grid searched to find optimal ...
2
votes
1answer
43 views

Can I use PCA after lasso variable selection?

I have a data regarding life satisfaction, of more than 2000 observations and 265 variables (most are categorical variables). I want to build a model, estimating the effects of society problems on the ...
1
vote
1answer
36 views

why does lasso select at most n predictors?

From the seminal paper on elastic net regularization from Zou and Hastie 2005, I read ...
0
votes
1answer
24 views

Application of LASSO , Ridge, PLS in feature selection of spectral data

The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based) But none of the features are removed ( no ...
0
votes
1answer
24 views

Logistic regression with coefficients penalized to other numbers

When you penalize logistic regression using l1, l2 or both penalizations, the coefficients are penalized towards 0. I would like to do the same thing but penalizing the coefficients towards other ...
0
votes
0answers
21 views

Additive Gaussian Processes with Penalized Likelihood

I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...
0
votes
0answers
29 views

Trouble understanding L1 and L2 cost function [duplicate]

When reading the Sklearn User Guide, one might see the following statement about Logistic Regression As an optimization problem, binary class L2 penalized logistic regression minimizes the ...
0
votes
0answers
28 views

Can you do LASSO with panel data that has cross-correlated errors?

suppose that there are observations of stocks return for firm i in 1..N date of observations t in 1..T The error structure is such that cov(epsilon_it,epsilon_jt) are far from zero. An important ...
3
votes
0answers
15 views

How sensitive is $L_p$ regression to initialisation?

Consider that I wish to solve a linear regression in the $L_p$ framework That is, the optimisation problem that I wish to solve is $$ \mathbf{w} = \text{argmin}_{\mathbf{w}} ||\mathbf{w}^T\mathbf{x} -...
2
votes
0answers
35 views

Consistency of Adaptive LASSO

I'm reading the paper on Adaptive LASSO estimator (Zou, 2006). In one of the presented numerical simulation examples (Model 0 (Inconsistent lasso path), page 6 (1423)) they claim the following: To ...
1
vote
1answer
34 views

Can I use Lasso directly in classification for feature selection?

In the scikit-learn package, Lasso is a linear regression model while it can be used for feature selection. However, is it reasonable if I use it directly in ...
0
votes
0answers
16 views

MultiTaskLasso vs. Lasso with dummies

I am trying to do a Lasso regression, where one of the features is a categorical string e.g. suppose we have Price,Year,Make for a car. One option would be to use one-hot encoding for Make, and do ...
0
votes
1answer
19 views

Question about Validation Set for hyperparameter tuning

Okay, I'm still a bit confused as to this Training/Validation/Test Set split. I might be wrong here, but from what I understand, the model is first applied to the Training set, to "learn" from it and ...
0
votes
0answers
16 views

How to give a represantation to veriables from each group using LASSO

I'm trying to apply LASSO regression on my data set in order to choose the best variables. However, my variables (44 to be accurate) come from 7 different groups, is there any option to give a "...
1
vote
0answers
60 views

Lasso-Regression on a casecohort with interval survival data (Cox-Barlow-Method)

My problem in a nutshell: I would like to do a lasso- regression in R on interval survival data with weights and there seems no package available which can handle lasso, interval survival data and ...
-1
votes
0answers
20 views

Variance Inflation Factor for Ridge Regression model

Is there such a thing as a metric that can determine if multicollinearity is violated in a ridge, elastic net, or lasso model? From programmatic terms, if I have a glmnet package model, is there a way ...
0
votes
1answer
36 views

how to use elastic net to select a set of features

I have a dataset with 500 samples and 100 features. I need to come up with a set of features. The management prefer a model with a smaller set of features. How exactly should I use elastic net to do ...
0
votes
0answers
37 views

Lasso and Sparsity

Consider the sparse regression problem $\theta^* = \operatorname*{argmin}_\theta \|X\theta -y\|_2^2 + \lambda\|\theta\|_1$ where $\theta \in \mathbb{R}^p$ and $X$ is of size $(n,p)$ Then, what if two ...
0
votes
1answer
32 views

Why can't ridge regression set slope to zero like LASSO does

I know that LASSO penalizes certain coefficents to zero by taking absolut value. However, ridge makes penalty by taking square instead. I am wondering why this difference forbid ridge from setting ...
0
votes
0answers
40 views

Finding the “optimal” non linear relationship between two variables

I am looking for finding associations between a binary outcome regarding women fertility and several potential risk factors. Since this study is quite exploratory, I was planning to include all my ...
0
votes
1answer
27 views

Does it make sense to run lasso to select features for neural network training?

I want to train a neural network for regression. $$\Bbb R^{2800} \rightarrow \Bbb R^{1}$$ The dimension of feature vectors is $2800$. The figure is an illustration of one of the feature vectors. ...
-1
votes
0answers
12 views

Beta regression and LASSO on multiply imputed data

As the title implies, I want to perform Beta regression with LASSO on a multiply imputed dataset. It seems to me that the general procedure for doing this should be a generalization of the group lasso ...
1
vote
0answers
25 views

Removing the intercept term for penalized logistic regression

I am working on lasso logistic regression and am trying to remove the intercept term from the penalty function. I have tried to use the mean centering theory but for logistic regression it can not be ...
2
votes
1answer
59 views

Large value of $X\beta$ in logistic regression?

In logistic regression, the probability is obtained from $$ Pr = \frac{\exp(X\beta)}{1 + \exp(X\beta)} ~~~~ (1) $$ From the plot below, it is obvious that if $X\beta$ > 10, the probability approaches ...
-1
votes
0answers
20 views

Asymptotics for Lasso-Type Estimators

In the paper Asymptotics for Lasso-Type Estimators, https://projecteuclid.org/euclid.aos/1015957397 the authors study the asymptotic properties of the LASSO estimators. I am confused, how can we ...
1
vote
0answers
42 views

Asymptotic properties for LASSO in logistic regression

I am trying to find a paper or book that gives the asymptotic properties for the LASSO for logistic regression. Anyone has any suggestions please?
0
votes
0answers
21 views

which parameter you choose on lasso CV, tuning parameter λ or βi constraint s?

I try to use lasso for prediction and I have $X_{tr} \subset X$ the train set and $Y_{tr}$ the train target. and I have $X_{ts} \subset X$ and $ Y_{ts}$ the test set for CV. I used CV and got $λ_i$ ...
0
votes
0answers
15 views

Predicting multi variable regression

I have a weekly sales data for a product which I have collected over past 16 years. Data is highly seasonal, cycles repeat themselves every 52 weeks. I am using python to build a forecasting model. ...
0
votes
0answers
53 views

LASSO regularization when p>n

When p>n, number of features greater than the number of observations, LASSO selects only n variables before saturates. What is the reason for this? Although I have read that the reason is the ...
0
votes
1answer
44 views

Penalize the intercept in lasso (L1) penalized logistic regression or not?

In logistic regression: $log(\frac{p(x)}{1-p(x)}) = \beta_0 + \beta_1x$, let $x' = \frac{x-\bar{x}}{\sigma_x}$, then in terms of the centered and scaled varaible $x'$ , $$ log(\frac{p(x')}{1-p(x')}) ...
0
votes
1answer
35 views

developing and assessing a prediction Cox model using lasso

I wonder if anyone can comment on if the following modelling strategy is valid please? I have a 200 patient survival data set (actually 2 data sets: 40 events and 160 events) and 100,000 ish candidate ...
1
vote
0answers
53 views

How does Ridge Regression penalize for complexity if the coefficients are never allowed to go to zero?

In the context of trying to understand regularization and how it works for ridge regression vs. lasso regression, I've come across two ideas: Both of these methods attempt to improve generalization ...
1
vote
0answers
11 views

margin point of view of logistic regression

I am currently studying Lasso for logistic regression and am using Buhlmann et.al book (Statistics for High Dimensional Data) to understand better. Section In this book on page 3.3.1 they define ...
2
votes
1answer
167 views

Why L1 regularization can “zero out the weights” and therefore leads to sparse models? [duplicate]

I'm aware there is a very relevant explanation on L1 regularization's effect on feature selection at here: Why L1 norm for sparse models [Ref. 1]. To better understand it I'm reading Google's ...
2
votes
0answers
55 views

Feature selection Stability of Elastic net vs Lasso

I am new to regularized regression, and I was told that Elastic net overcomes many issues of the Lasso Regression. Especially, in the case of highly correlated predictors, Lasso variable selections ...
7
votes
2answers
63 views

I regularized my linear regression, now what?

I have estimated the regression parameters of a linear regression models using LASSO, sent some variables to zero using cross validation, and now I got a final model. It is known that regularizing ...
1
vote
1answer
79 views

Regularizing the inverse coefficient matrix in multivariate regression

I'd like to minimize the objective $ \operatorname{tr}[ (Y-XR^{-1})^T (Y-XR^{-1}) ] + \lambda \sum_{ij} |R_{ij}|$ wrt to $R$ (which is $P \times P$ but non-symmetric) where $Y$ and $X$ are both $N \...
3
votes
3answers
76 views

For high dimensional data, does it make sense to do feature selection before running elastic net?

I have a dataset with $n = 800$ observations and $p = 2000$ features. I'm running elastic net for binary classification. My question is: Does it make sense to do some feature selection to reduce the ...
0
votes
0answers
81 views

Large Matrix to run cv.glmnet for multinomial

I am working on a large matrix with number of samples N=40 and features, P=7130. I am trying to fit the cv.glmnet() for the ridge but i am getting error while doing ...
1
vote
1answer
94 views

Mean squared error (MSE) prediction performance: Ridge vs Lasso?

It says that the ridge will outperform lasso in terms of prediction performance when the prediction metric is MSE, according to the answer to this post below: If only prediction is of interest, why ...
2
votes
1answer
40 views

Expected value of residuals for LASSO model?

For simple OLS models the expected value of the residuals E(ϵ)=0 can be shown to be zero if an intercept is included in the regression equation. I am using a LASSO model and was wondering if the ...
0
votes
0answers
12 views

Are there any plots for the results of the Lasso estimator besides plotting the Lasso path?

When one reports the results of methods like Lasso, group Lasso or Stability Selection, are there any nice plots one could generate for genome-wide association studies (besides lasso paths) to make ...
1
vote
1answer
30 views

Is Elastic Net my best choice for finding sparse linear models in correlated features?

I have a linear regression problem, 1000 data points, but with 36 correlated features, those features are very highly correlated. And I know the ground truth must be linear. I know Lasso would give ...