A form of regularization used in the estimation of regression coefficients that shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). Some coefficients may be shrunk to zero; thus the LASSO performs variable selection. The LASSO is equivalent to the ...

learn more… | top users | synonyms

0
votes
1answer
48 views

Help interpret cv.glmnet behavior for very large set of potential predictors

I've created this toy example to demonstrate something that is occurring with my real data. I do not understand how to interpret the apparent failure of cv.glmnet to find a solution when it's ...
0
votes
0answers
24 views

What is multi run lasso regression?

I have problem in understanding of multi-run lasso regression. Basically, I know what is lasso regression, but don't know what is multi-run lasso regression, which sometimes I see literatures. Does ...
2
votes
0answers
22 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
0answers
19 views

Parameter tuning in lars (lasso) matlab

I am trying to use lars (matlab implementation:http://www.ece.ubc.ca/~xiaohuic/code/LARS/LARS.htm). I want to do a leave one out cross validation on my data using this code. I have the following ...
7
votes
2answers
127 views

Selection of k knots in regression smoothing spline equivalent to k categorical variables?

I'm working on a predictive cost model where the patient's age (an integer quantity measured in years) is one of the predictor variables. A strong nonlinear relationship between age and risk of a ...
1
vote
2answers
72 views

Strict convexity of Ridge vs Convexity of LASSO

Is there any intuition why the ridge regression is strictly convex, while the LASSO is only convex? Does it have to do with the "corners" of the L1 regularization?
2
votes
0answers
18 views

How can I tell if I my sample size is large enough for reliable feature selection using LASSO regression?

I have a gene expression dataset with 20 samples, and am not going to be getting any more. There are ~28,000 genes and four clinical covariates associated with each sample. The gene expression values ...
3
votes
1answer
141 views

Ridge, lasso and elastic net

How do ridge, LASSO and elasticnet regularization methods compare? What are their respective advantages and disadvantages? Any good technical paper, or lecture notes would be appreciated as well.
1
vote
1answer
84 views

Why no mention of penalized regression techniques in Applied Logistic Regression, 3rd edition, by Hosmer, Lemeshow, and Sturdivant?

Just ordered this textbook, and Wow, the complete omission of this subject from an otherwise excellent reference on logistic regression is a bit surprising. The 2nd edition was published in 2000 - ...
1
vote
0answers
21 views

scikit learn: add lasso or ridge penalty only on subset of parameters

Is there a way of using the linear model api to add the lasso penalty for a subset of the parameters I am regressing? For example, consider a linear separable decomposition that I want to fit to some ...
24
votes
6answers
296 views

Standard errors for lasso prediction using R

I'm trying to use a LASSO model for prediction, and I need to estimate standard errors. Surely someone has already written a package to do this. But as far as I can see, none of the packages on CRAN ...
0
votes
1answer
114 views

Logistic regression using penalized likelihood (lasso?) in Matlab/R

I am trying to use logistic regression in a scenario where there are very few positives. I'm aware that maximum likelihood suffers from small sample bias. So ...
6
votes
1answer
80 views

KKT versus unconstrained formulation of lasso regression

L1 penalized regression (aka lasso) is presented in two formulations. Let the two objective functions be $$ Q_1 = \frac{1}{2}||Y - X\beta||_2^2 \\ Q_2 =\frac{1}{2}||Y - X\beta||_2^2 + \lambda ...
0
votes
1answer
79 views

Superiority of LASSO over forward selection/backward elimination in terms of the cross validation prediction error of the model

I obtained three reduced models from a original full model using forward selection backward elimination L1 penalization technique (LASSO) For the models obtained using forward selection/backward ...
2
votes
1answer
75 views

cv.glmnet lambda stability

Intro I am running cv.glmnet from the glmnet package in R. I am running 10-fold cross ...
1
vote
0answers
28 views

Implementation of generalized cross-validation for constrained LASSO problem

I have a non-negativity constrained LASSO problem like this: min: $||Cx-b||_2^2 + \lambda||x||_1$ subject to: $x\geq0$ where C is a matrix, and x and b are column vectors. Now I want to ...
1
vote
0answers
49 views

Can the predicted value vs observed value plot have a slope not equal 1 in a LASSO model?

I was trying to use glmnet package in R to create a lasso regression model. The details of my data are: Dependent variables $y$: 451 observations, single value for each observation. Independent ...
3
votes
2answers
49 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
3
votes
1answer
56 views

Interpreting the lasso coefficients

I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features. I want to know based upon the coefficient values how do I rank the features?
4
votes
1answer
54 views

Averaging LASSO coefficients for repeated random partitioning of data

Is it reasonable to average LASSO coefficients from repeated reshuffling of training/test sets? Suppose I randomly divide my data into testing & training sets, then within the training set use ...
4
votes
1answer
117 views

Is standardisation before Lasso really necessary?

I have read three main reasons for standardising variables before something such as Lasso regression: 1) Interpretability of coefficients. 2) Ability to rank the ...
0
votes
0answers
20 views

Active set methods for group lasso?

When the solution is extremely sparse, probably the fastest method for solving LASSO regression is least angle regression, which starts from an all-zero solution and adds nonzero elements to the ...
0
votes
1answer
117 views

LASSO to identify important variables in ordered logistic regression?

I have spent two days grappling with this question, and the range of ambiguous answers online has driven me to ask. I am working with R. I have a dataset where my dependent variable is an ordered ...
1
vote
1answer
117 views

Fastest way to run ridge regression on large datasets where n>>p

Provided that you don't want to do any variable selection: Is there any software which is faster than glmnet at vanilla ridge regression for large datasets?
2
votes
0answers
40 views

Standardize continuous predictor variables on [0, 1] scale?

I'm working on a health care regression model predicting # of inpatient visits. My analysis dataset includes a number of hybrid continuous/categorical predictor variables which can hold values on a 0 ...
1
vote
2answers
71 views

Linear regression for time-series prediction

Say we have $N$ time series $X_t^i$ for $i=1...N$and we want to predict a separate time series $Y_t$. Let's consider the following model: $Y_t = \sum_{i} \beta_i X_{t-1}^i $ I am just trying to ...
2
votes
0answers
37 views

Lasso for GEE model

Can a LASSO be applied for predictor selection in a logistic GEE (generalized estimating equations) model for longitudinal data? Is there an implementation of LASSO for a logistic GEE model for ...
8
votes
2answers
225 views

Why use Lasso estimates over OLS estimates on variable subset?

For Lasso regression $L(\beta)=(X\beta-y)'(X\beta-y)+\lambda*norm(\beta,1)$, suppose the best solution (minimum testing error for example) selects $k$ features, so that ...
6
votes
4answers
160 views

How to fix one coefficient and fit others using regression

I would like to manually fix a certain coefficient, say $\beta_1=1.0$, then fit coefficients to all other predictors, while keeping $\beta_1=1.0$ in the model. How can I achieve this using R? I'd ...
0
votes
0answers
11 views

A little problem to achive this by coding in convex constraint

IN convex optimization : min f(x), I have two constraits : one is sum_i|x_i| <= t (t can be any numbers) , another constraints is |x_j|=|x_j-1 k_j|(set k_1 can not equal to zero and k_j can be ...
5
votes
1answer
74 views

Empirical justification for the one standard error rule when using cross-validation

Are there any empirical studies justifying the use of the one standard error rule in favour of parsimony? Obviously it depends on the data-generation process of the data, but anything which analyses a ...
0
votes
0answers
36 views

No minimum in cross validation error

I am evaluating both non-linear LASSO and SVM for a high-dimensional non-linear dataset. In order to do that I am applying 10-fold cross validation. The training error decreases as usual as the model ...
0
votes
0answers
62 views

How to sample from this dirichlet distribution with an L1 prior?

I'd like to draw a sample from a distribution with p.d.f $$f(p,q,r,s) \sim \mathrm{e}^{-w(|p+q-r-s|+|p-q-r+s|+|p-q+r-s|)}p^aq^br^cs^d \mathbb{1}_{p+q+r+s=1}$$ $w > 0$ is a free parameter (which ...
4
votes
1answer
209 views

How to interpret the lasso selection plot [duplicate]

I did lasso selection using lars::lars(), then I got this plot. I have no idea how to interpret it: Could anyone provide a brief explanation? Why does it plot ...
3
votes
1answer
118 views

LASSO with L1 loss function

I've been trying to figure out a way to perform LASSO with L1 loss function (instead of the L2 loss) but have been completely dumfounded as to how. I've attempted to use the flare package's ...
0
votes
0answers
31 views

Sample size for LASSO

What considerations should be made regarding sample size for using LASSO or elasticnet? We are going to gather expression data from 1700 genes and our response variable is multinomial (3 categories).
16
votes
4answers
432 views

When wouldn't I use LASSO for model selection?

Assume that you need to build a linear model to make predictions for new observations, and that there is uncertainty about which subset of variables should be included in the model. You are only ...
1
vote
1answer
125 views

How is $\lambda$ tuning parameter in lasso logistic regression generated

I know glmnet(x,y) generates $\lambda$ but I am very curious to know the actual formula that is behind this, generating $\lambda$.
0
votes
2answers
63 views

Positively constrained parameters in LAD LASSO

I am looking for an implementation of LAD LASSO method with constraints on coefficients, such as non-negative beta coefficients for example. $\min_{\beta} \left| \mathbb{y} - \mathbb{X} ...
0
votes
0answers
13 views

Why the maxStages argument in biglars.fit does not work

Why doesn't the biglars.fit function work when maxStages is specified? I've tried multiple values and multiple ways of casting $y$ but it doesn't work. ...
2
votes
1answer
161 views

What is the time complexity of Lasso regression

What is the asymptotic time complexity of Lasso regression as either the number of rows or columns grows?
1
vote
0answers
46 views

Standardizing response variable in shrinkage/regularization

I know that I should standardize my predictors before estimating something like Lasso. But what about the response variable? Do I standardise this as well? Only ...
0
votes
0answers
30 views

How to add a non-negative constraint to lasso4j?

Lasso4j is a Java implementation of the Lasso L1-constrained fitting for linear regression. I would like to add a non-negativity constraint on the weights, meaning that the non-zero sparse ...
2
votes
0answers
27 views

Lasso-ing the order of a lag?

Suppose I have longitudinal data of the form $\mathbf Y = (Y_1, \ldots, Y_J) \sim \mathcal N(\mu, \Sigma)$ (I have multiple observations, this is just the form of a single one). I'm interested in ...
2
votes
1answer
53 views

Interpretation of lasso recovery results

When people say that lasso regression can under certain assumptions recover "the support", i.e. non-zero regression weights, what does this mean? This cannot mean causal recovery, because Pearl has ...
12
votes
2answers
396 views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
1
vote
0answers
110 views

Pros and Cons of different types of regression? [closed]

I'm currently studying Linear Regression and am wondering if somebody would help me out with a high-level comparison of different methods. Just to list some: Lasso, Ridge, Elastic Net, Principal ...
3
votes
0answers
120 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
2
votes
2answers
75 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in ...
2
votes
0answers
150 views

Lasso ||a|| and “General Lasso” ||Da||

Ryan Tibshirani introduced once a more general type of Lasso, where the regularizer is $$\parallel D \alpha \parallel_1$$ instead of $\parallel \alpha \parallel_1$. See paper However, there is nearly ...