A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

learn more… | top users | synonyms

0
votes
0answers
15 views

Autoencoders & Predictive sparse decomposition (PSD) & Alternating Direction Multiplier Method (ADMM)

I am studying Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville. In Chapter #14 Autoencoders the authors write Internally, it has a hidden layer $h$ that describes a code used to ...
2
votes
1answer
30 views

Lasso and Ridge tuning parameter scope

In ridge and lasso linear regression, an important step is to choose the tuning parameter lambda, often I use grid search on log scale from -6->4, it works well on ridge, but on lasso, should I take ...
0
votes
0answers
6 views

Multiple variables / continuous outcome / model formula

I have a set of continuous / discrete variables with which I want to model a continuous outcome. How can I know which type of curve would be a good choice and, therefore, which kind of function to ...
0
votes
0answers
7 views

Iterating Lasso

Can Lasso regression be performed multiple times to systematically clean/remove parameters from a model? Would there be downsides to doing so/would it be considered poor practice? Thanks!
0
votes
0answers
6 views

Unbalanced binary features in LASSO regression

I have a target $y$ that I want to predict from variables $x_1, x_2, \ldots x_k$. Suppose the first of these variables, $x_1$, is a binary variable (i.e., only taking on one of 2 values). If I use ...
0
votes
0answers
8 views

Lasso eliminating features but then they reappear?

Does there exist a set of samples and outputs such that running lasso with some value $\lambda$ for the $\mathcal{L}_1$ penalty zeros out some coefficient $w_i$ but for some $\lambda' > \lambda$, ...
1
vote
1answer
21 views

Selecting a loss-function for k-fold cross-validation over shrinkage parameter

I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $\lambda$ on the basis of cross-validation (CV). In this case, I have 50k ...
0
votes
0answers
28 views

Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
3
votes
0answers
28 views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
0
votes
0answers
27 views

Obtain lasso regression coeficient based LS when $X'X = I$

I need to obtain coefficients of lasso regression based in coefficients of Least Square regression method when $X'X = I $. any help will be appreciated.
0
votes
0answers
8 views

fast way to train a classifier on different but overlapping features

I am training a linear classifier repeatedly on different set of overlapping features. I have a 3D grid of features, each time features from a small sphere from a grid are used to train a classifier, ...
0
votes
0answers
42 views

Can L1 linear regression perform worse than vanilla linear regression on fewer features?

I have a data set with 2 features and I'm trying to predict one real-valued variable. I use linear regression and I measure the error using 10-fold CV and absolute mean error as a metric. I noticed ...
1
vote
1answer
34 views

What is the relationship between regression analysis, LASSO, and coordinate descent?

I'm a complete newbie and trying to understand what exactly LASSO is, how coordinate descent is used with LASSO, and how all of that factors into regression analysis. I'm totally confused about the ...
6
votes
1answer
111 views

LASSO regularisation parameter from LARS algorithm

In their seminal paper 'Least Angle Regression', Efron et al describe a simple modification of the LARS algorithm which allows to compute full LASSO regularisation paths. I have implemented this ...
0
votes
0answers
15 views

weighted lasso/enet using lqa package in R

I'm using lqa to solve my lasso model. However, I need to define a penalty weight/factor which ranges between 0 and 1. 0 shows no penalty has to be applied and 1 means that the lasso penalty has to be ...
0
votes
0answers
17 views

elastic net regression with hierarchal constraints for higher order interactions

I've been using heirNet and glinternet to preserve hierarchy constraints for pairwise interactions in my model, does anybody know if these methods can be extended to 3 way (or higher order) ...
1
vote
0answers
42 views

Lasso Regression - model predictions are not correct. low r-squared

I am attempting to use Lasso to choose the best variables from a set of 20. I have managed to construct a model using LassoCV, however when using the test data to compare the predicted returns to the ...
0
votes
1answer
46 views

Lasso Regression - Finding multiple candidate models

I have 20 predictors and I am attempting to find several candidate models to then test. I am using the LassoCV library, my following code provides me with the alpha and co-efficients of a model. ...
1
vote
1answer
39 views

Standardized LASSO in R still has intercept

I understand the need to standardize variables when performing LASSO in R (I'm specifically using cv.glmnet, and setting ...
1
vote
1answer
38 views

cross validation after lasso

I used cross validation to select lambda. Then I performed lasso and get non zero coefficients (features). Shall I perform cross validation for these non zero coefficients as a kind of validation?
3
votes
1answer
30 views

Interpreting Special Case for Ridge Regression and the Lasso

The below text is from Statistical Learning Page no.225 Consider a case with $n = p$, and $\mathbf{X}$ a diagonal matrix with 1’s on the diagonal and 0’s in all off-diagonal elements. To simplify ...
8
votes
1answer
177 views

Variable coefficient rises, then falls as lambda decreases (LASSO)

I am regressing a continuous predictor on over 60 variables (both continuous and categorical) using LASSO (glmnet). In examining the variable trace plot, I notice that as log lambda increases, one of ...
1
vote
0answers
24 views

Defining Importance of variables in regression and variable selection

When doing variable selection, one of the most asked questions is which variables are most important, or rank the variables in order of importance. Typically in linear or logistic regression, the ...
0
votes
1answer
33 views

Ordinal outcome survey regularization and variable selection

I am analyzing some survey data with many thousands of respondants. My main dependent variable of interest is a three-level Likert scale (very/pretty/not-so-much). I have ~50 predictors. I would like ...
2
votes
0answers
28 views

Is it possible to use LASSO regression with multi-levlel data?

I have real-time monitoring data where participants report on a variety of variables four times per day for a month. Is it possible to use LASSO regression (e.g,. glmnet r package) with this data? I'm ...
0
votes
0answers
23 views

Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
4
votes
1answer
64 views

How to perform non-negative ridge regression?

How to perform non-negative ridge regression? Non-negative lasso is available in scikit-learn, but for ridge, I cannot enforce non-negativity of betas, and indeed, ...
0
votes
0answers
19 views

Difference of feature importance from Random Forest and Regularized Logistic Regression

I have 13 features in a classification task and I use Random Forest, L1 logistic regression and L2 logistic regression for as separate classifiers and would like to compare their performance. Although ...
0
votes
0answers
10 views

What do eps and tol do in LassoCV (scikit-learn)

Using scikit-learn: http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LassoCV.html Specifically, I am interested in: 1) If eps grows, does the accuracy(precision) increase or ...
1
vote
0answers
17 views

what is the close form expression for the update of $x_i$ in coordinate descent?

I am trying to derive the close form expression for the update of $\omega_i$ in coordinate descent for lasso, but I find it hard to proceed. Could you give me some hints? ...
1
vote
0answers
17 views

Bootstrap on a weighted subset

My group is developing an ML algorithm. We have a number of validation datasets we use to compare variants of the algorithm, but the validation is expensive in the sense that to test $m$ variants on ...
0
votes
1answer
55 views

Is this model the Lasso model?

The following model is about optimization problem with restrictions, The goal is to find the optimal solution of matrix W. I want to know that is this model the ...
1
vote
0answers
45 views

How is $\alpha$ chosen in lasso? [closed]

I am looking at the scikit_learn (python) implementation of LassoCV. Below is the function that sets the $\alpha$ values to look at (the penalization coefficient of the L1 norm). Can anyone shed ...
0
votes
0answers
33 views

Lasso does not converge

I have a 2000x100 matrix (100 features, 2000 observations). I am trying to do a simple lasso regression (L1 regularization). However, the algorithm does not converge (I use lasso of sckit_learn in ...
2
votes
2answers
59 views

Lasso with constraint on some coefficients (not all)

I would like to run a lasso regression (L1 penalisation) with a twist: there are different constraints on my problem. The coefficients for my features (predictors) are $\beta_i$. I want to find the ...
0
votes
0answers
59 views

A better understanding of Lasso

I am a little new to using lasso regression, and I wanted to know some more about its relationship in determining potential interactions among terms. From my understanding Lasso determines the ...
6
votes
1answer
399 views

Is regression with L1 regularization the same as Lasso, and with L2 regularization the same as ridge regression? And how to write “Lasso”?

I'm a software engineer learning machine learning, particularly through Andrew Ng's machine learning courses. While studying linear regression with regularization, I've found terms that are confusing: ...
0
votes
0answers
21 views

find non linear dependencies in the model automatically with Lasso

I have a little knowledge on Lasso, from what i know its pretty good at feature selection and also finds the best sweet spot between bias and variance trade-off. If we are to come up with a regression ...
2
votes
1answer
67 views

GBM and highly correlated predictors

I have a data set with 70 predictors, 68 numeric and 2 factors. When I build a gbm model using all predictors, I get an R2 of 0.767 and RMSE of 175.15. I get similar numbers on the training and the ...
5
votes
1answer
266 views

Why I am that unsuccessful with predicting with Generalized Lasso (genlasso {genlasso})?

I try to utilize Generalized Lasso genlasso {genlasso} function to incorporate additional penalty matrix into estimation process. I started with a "hello world" ...
3
votes
0answers
83 views

Graphical LASSO Interpretation

I have a conceptual question about graphical LASSO interpretation. I have been using the huge package in R to estimate an association network for a matrix of node ...
0
votes
0answers
31 views

Lasso - Univariate Analysis?

I am using the lasso to perform variable selection (logistic regression) but it is usually the norm to first report univariate odds ratio/significance prior to performing variable selection. However, ...
2
votes
0answers
44 views

When does LASSO regression using coordinate descent fail to converge?

I am fitting a large dataset using the glmnet package in R. My response is binomial and I have substituted the covariates with weight-of-evidence to make them all continuous and all on the same scale. ...
1
vote
1answer
69 views

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as ...
0
votes
0answers
26 views

lasso gibbs sampler

Hi guys I tried to build a gibbs sampler for lasso regression. here is the model $y = Normal(x\beta,\sigma)\\ \beta = Laplace(alpha)\\ \sigma = Gamma(a,b)\\ \alpha = Gamma(c,d) $ I just used a ...
0
votes
1answer
60 views

glasso- Assumptions of Meinhausen-Buhlmann approximation?

First off, This is not an R question - it is a conceptual question, so there is no need to perform any code. Problem: I'm trying to invert a large dimensional covariance matrix of p features. For ...
1
vote
0answers
25 views

Inverse covariance estimation for generalised Lasso

I have implemented Lasso for estimating sparse inverse covariance case using ADMM in Matlab. Inverse covariance estimation using LASSO regularisation, X is the estimate, S is empirical covariance ...
4
votes
1answer
96 views

Bias of Tibshirani's Lasso estimator

I am searching for a theorem that gives upper bounds for the bias of the Lasso estimator from Tibshirani[1]. Do anybody know such a theorem? [1] Tibshirani, R., (1996). “Regression Shrinkage and ...
7
votes
1answer
91 views

What are the pros and cons of employing LASSO for causality analysis?

It looks like social sciences are impressed by Statistical Learning and its results. A couple of months ago, I heard Imbens saying: "LASSO is the new OLS". My problem with this is that I've been ...
1
vote
0answers
20 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...