Is a form of regularization used in estimation of regression coefficients which shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). The LASSO is equivalent to the Bayesian estimation problem where iid standard Laplacian prior is used for the ...
0
votes
0answers
21 views
Upper bound for the Cardinal of the Active Set in Lasso
I am searching for a proof to:
With probability tending to one, we have that the cardianl of the active set in the Lasso estimator is bounded by $C\frac{n}{\lambda_n^2}$, ($C$ is some constant) i.e.;
...
0
votes
0answers
17 views
Estimating sparse inverse covariance matrix in high dimensional data
I am trying to estimate the graph in very high dimensional data, I mean with million nodes. Up to now all the papers that I have found, they are limited to few thousands.
All of them like graphical ...
2
votes
0answers
40 views
Why isn't the Dantzig selector popular in applied statistics?
Lasso-like methods have become pretty common in applied statistics but the Dantzig selector remains unpopular despite having great properties (minimax optimality). Why hasn't it become more popular?
2
votes
2answers
130 views
Using LASSO from lars (or glmnet) package in R for variable selection
Sorry if this question comes across a little basic.
I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
0
votes
0answers
32 views
lasso and cross-validation (theoretical results)
is there any theoretical result which says that use the minimum of the cross-validation as value for the lasso penalty is a good choice?
I would like something like $P(S_0 \subset \hat ...
0
votes
1answer
21 views
Loss sequence for glmnet
Is the loss sequence returned by cv.glmnet (the slot "cvm") on the standardized variable scale or the original scale? This only affects the penalty part of the loss, I think. In other words, are ...
2
votes
0answers
33 views
How to report most important predictors using glmnet?
I want to find the most important predictors for a binomial dependent variable out of a set of more than 43,000 independent variables (These form the columns of my input dataset). The number of ...
0
votes
0answers
24 views
Coding an adaptive elastic net in Matlab
I've coded an adaptive lasso based on Zou 2006 and I'm in the process of figuring out how to implement the adaptive elastic net.
The L1 penalization has associated weights $w_j$, proportional to the ...
-2
votes
1answer
48 views
LASSO Estimator
Could you help me to find some explications why the LASSO is better in selection, whereas, it is so bad on estimation.
In other words what are the limits of the LASSO?
Thank You !
2
votes
1answer
59 views
OLS result with LASSO
I'm using the lasso function in MATLAB to perform regularized regression for parameter selection. I'm new to LASSO, so I generated some toy data and tested LASSO with lambda = 0 to make sure I would ...
3
votes
0answers
65 views
How does LASSO select among collinear predictors?
I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection.
...
2
votes
0answers
102 views
Using LASSO on random forest
I would like to create a random forest using the following process:
Build a tree on a random samples of the data and features using information gain to determine splits
Terminate a leaf node if it ...
3
votes
1answer
127 views
LASSO vs forward selection
I have two questions:
I use cross validation to select a LASSO model, does the step in which a particular variable enter, indicate its relative importance?
Let's age enter in step 1 and sex enter in ...
1
vote
2answers
111 views
Are penalized regression techniques greedy algorithms?
In other words, is it feasible any of the various penalized regression techniques (such as ridge regression, lasso, and elasticnet) could completely miss the optimal solution for a regression model ...
2
votes
1answer
102 views
What is deviance in lassoglm
I am trying to fit a lasso penalized logistic regression model to a certain data. I am using lassoglm for that in matlab. I use the following function
[B,FitInfo] = ...
2
votes
1answer
64 views
Confusion related to the cross-validated deviance of lasso fit
I am having a certain confusion related to cross-validated deviance of lasso fit. I am not sure what is being done. Lets say I run lassoglm for my dataset having 1000 examples and 15 features with a ...
0
votes
0answers
46 views
Confusion while using lassoglm
I am trying to fit a logistic regression model with L1 regularization on my data. My data has just 12 examples with 150 features. So I used L1 regularization. Now when I use the lassoglm function like ...
2
votes
0answers
116 views
LASSO vs AIC for feature selection with the Cox model
I have some questions about the Lasso.
After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
83 views
Can the bias introduced by lasso change the sign of a coefficient?
L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would ...
0
votes
1answer
117 views
Using LASSO to rank features
Suppose I want to answer the following question:
Rank the explanatory variables from most important to least important.
I want to do the following:
Pick lambda from largest to small, and report in a ...
1
vote
0answers
264 views
How to select the best variables by RandomForest in R?
I have a table of mRNA levels of my target gene and it's transcription factors in many different condition. What I want to do is to select the most important ...
1
vote
0answers
107 views
Bootstrap randomized Lasso selection for a Cox model
I'm interested in variable selection for a cox proportional hazards model.
I've read this article which slightly favours randomized bootstrap lasso selection over bootstrap lasso selection since it ...
3
votes
0answers
102 views
Dantzig Selector, LASSO, LAD LASSO
I am wondering about this. When is it best to use Dantzig Selector (the infinity normed error measure plus the L1 regularizer) , the LASSO (the mean square error measure plus the L1 regularizer), and ...
1
vote
2answers
239 views
Using sparse inverse covariance matrix in estimating least squares coefficients
I am reading the paper introducing the graphical lasso, which is a way to estimate a sparse inverse covariance matrix.
http://www-stat.stanford.edu/~tibs/ftp/graph.pdf
Finding a sparse inverse ...
2
votes
0answers
200 views
When using glmnet how to report p-value significance to claim significance of predictors?
I have a large set of predictors (more than 43,000) for predicting a dependent variable which can take 2 values (0 or 1). The number of observations is more than 45,000. Most of the predictors are ...
4
votes
1answer
87 views
Logistic regression without negative samples
I have a data set of RNA reaction values of breast cancer. I want to figure out which RNAs are essential genes by Logistic Regression & LASSO.
The data set has no negative samples. What should I ...
4
votes
0answers
115 views
Logistic regression model for analysis of many IVs with a relatively small sample size
I'm trying to determine the influence (direction and relative strength) of certain attributes of incoming students to an academic program on their successful completion of the program. My sample size ...
1
vote
0answers
115 views
Kernel in PenalizedSVM R package
There is not option to select kernel in penalizedSVM R package. What kernel do they use?
Is there some other R package with penalized SVM methods where I can choose various kernels?
0
votes
0answers
46 views
Order of variable entry in LASSO
It seems that the order of variable entry into a lasso model is significantly important! Is there any way that we make our model more meaningful? I mean is that OK to order the variables using another ...
1
vote
0answers
85 views
Derivation of closed form lasso solution: KKT optimality conditions
As defined in this thread Derivation of closed form lasso solution I would like to see a derivation of the closed form soft-thresholding solution for the LASSO using the KKT optimality conditions. ...
4
votes
1answer
175 views
LARS - LASSO with weights
I am interested in solving the following problem
$$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + ...
0
votes
0answers
56 views
How in every bootstrap sample value of $\lambda$ can be selected?
My assignment is to obtain bootstrap samples from prostate data and from each bootstrap sample find the value of tuning parameter.
3
votes
1answer
316 views
Stepwise regression vs. elastic net
I understand that StepWise regression analysis has lots of limitations, including the assumption that the predictors are not highly correlated with each other. In fact, this limitation was the most ...
0
votes
1answer
56 views
Using standardized Y in Elastic Net
I have an Elastic Net model that is selecting a number of variables from X, for prediction of Y. The assumption for Elastic Net is that X is standardized (I'm using Z-Scores), and Y is centered around ...
1
vote
0answers
128 views
Calculating R-square for Elastic Net
I am trying to do "variable selection" using Elastic Net method (Matlab Lasso function with alpha of 0.5). I have 75 predictors in total (some are correlated with each other, hence using Elastic Net ...
3
votes
2answers
236 views
Non negative lasso implementation in R
I am looking for some open source or an existing library I can use. As far as I tell the glmnet package isn't very easily extensible to cover the non negative case. I may be wrong, Any one with any ...
5
votes
1answer
276 views
If p > n, the lasso selects at most n variables
One of the motivations for the elastic net was the following limitation of LASSO:
"In the p > n case, the lasso selects at most n variables before it saturates, because of the nature of the convex ...
9
votes
2answers
207 views
Advantages of doing “double lasso” or performing lasso twice?
I once heard a method of using the lasso twice (like a double-lasso) where you perform lasso on the original set of variables, say S1, obtain a sparse set called S2, and then perform lasso again on ...
3
votes
0answers
121 views
Variable Selection One by One vs Simultaneously
The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
6
votes
1answer
3k views
How to present results of a Lasso using glmnet?
I would like to find predictors for a continuous dependent variable out of a set of 30 independent variables. I am using Lasso regression as implemented in the glmnet package in R. Here is some dummy ...
5
votes
1answer
382 views
L1 regression estimates median whereas L2 regression estimates mean?
So I was asked a question on which central measures L1 (i.e., lasso) and L2 (i.e., ridge regression) estimated. The answer is L1=median and L2=mean. Is there any type of intuitive reasoning to this? ...
4
votes
1answer
324 views
Variable selection with LASSO
I am trying to fit a predictive gene-based model in survival analysis. My question is:
Can I use LASSO as a variable selection method, and then run a multivariate Cox regression to get the ...
8
votes
2answers
829 views
Why do Lars and Glmnet give different solutions for the Lasso problem?
I want to better understand the R packages Lars and Glmnet, which are used to solve the Lasso problem:
$$min_{(\beta_0 \beta) ...
0
votes
1answer
386 views
How to use glmnet without the intercept?
I know the glmnet() function cannot exclude the intercept by users, but does anyone know how to derive the fit without intercept from the ...
6
votes
1answer
123 views
Ridge and LASSO given a covariance structure?
After reading Chapter 3 in the Elements of Statistical Learning (Hastie, Tibshrani & Friedman), I wondered if it was possible to implement the famous shrinkage methods quoted on the title of this ...
7
votes
2answers
410 views
How defensible is it to choose $\lambda$ in a LASSO model so that it yields the number of nonzero predictors one desires?
When I determine my lambda through cross-validation, all coefficients become zero. But I have some hints from the literature that some of the predictors should definitely affect the outcome. Is it ...
4
votes
1answer
315 views
Gradient descent and elastic-net logistic regression
I'm currently in the process of trying to understand the paper Regularization Paths for Generalized Linear Models via Coordinate Descent by Friedman et al. with regard to the regularization of ...
3
votes
0answers
238 views
When does LASSO select correlated predictors?
I'm using the package 'lars' in R with the following code:
...
3
votes
1answer
259 views
Standard error of parameter estimates in regularized regression
In a regularized linear regression model (e.g., ridge regression, lasso, etc.), what is the best way to obtain standard errors for parameter estimates? If cross-validation is used, is it ...
5
votes
1answer
444 views
regularized bayesian logistic regression in JAGS
There are several math-heavy papers that describe the Bayesian Lasso, but I want tested, correct JAGS code that I can use.
Could someone post sample BUGS / JAGS code that implements regularized ...
