Questions tagged [lasso]

A regularization method for regression models that shrinks coefficients towards zero, making some of them equal to zero. Thus lasso performs feature selection.

Filter by
Sorted by
Tagged with
1 vote
0 answers
14 views

variance estimation for fussed lasso

In the paper THE SOLUTION PATH OF THE GENERALIZED LASSO the authors derived degrees of freedom for the generalised lasso. Assume that $y\in\mathcal{N}(\mu,\Sigma)$, with $\mu\in\mathbb{R}^{n}$ and $\...
user avatar
  • 426
0 votes
0 answers
25 views

scaling features for LASSO variable selection

I am interested in performing LASSO regression for the purpose of variable selection. The response variable is categorical (3 classes) and most of the predictor variables are categorical. Most ...
user avatar
  • 139
0 votes
0 answers
9 views

Why do we take the maximum probability to define stable variables in stability selection?

In stability selection (link), we first calculate the selection probabilities, $\Pi_K^\lambda$. Then, using these probabilities, the stable variables are defined as $$S^\text{stable} = \{ k: \max(\...
user avatar
1 vote
0 answers
45 views

Out-of-sample prediction error using nested cross-validation

I am applying Lasso regression and the R function glmnet::cv.glmnet() to obtain a prediction model based on 90% of the data. I have set aside 10% as a hold-out set and obtain predicted probabilities ...
user avatar
  • 11
0 votes
0 answers
13 views

Small n - dimension reduction and regression

I am trying to develop a valid method for modeling some biological data. I do not need it to be predictive necessarily, but I want to be able to show relationships and determine which set (of many) ...
user avatar
0 votes
0 answers
17 views

Glment for lasso multinomial regression with all categorical predictors

I'm new to the community, so sorry for the mistakes I might make in writing my question. My question is both related to statistics and coding, because I am not entirely sure that the method I came up ...
user avatar
0 votes
0 answers
13 views

glmnet with weighted penalty

I need to fit a elastic net penalized logistic regression model in the form of Here W is a positive definite weight matrix. Since ...
user avatar
  • 283
0 votes
0 answers
14 views

How to identify variables with prognostic predictive value?

We have collected biopsy data and clinical data from 232 patients. The are in total 45 different clinical and histopathological variables available. What statistical method(s) is recommended to ...
user avatar
0 votes
0 answers
16 views

How does adding a predictor lead to a higher covariance between prediction and actual values

I am trying to figure out what the best predictors are for a certain variable. I used gglasso to get the best predictors, resulting in two groups (two variables ...
user avatar
  • 239
0 votes
0 answers
10 views

R: Plotting lambda values and coefficients in lasso regression

I am running lasso and got a graph of log lambda vs coefficients by running: plot(lasso$finalModel, xvar="lambda", label=T) The graph has 1,2,3, etc. for the variables. How do I identify ...
user avatar
  • 1
2 votes
0 answers
31 views

LASSO and duality theorem

I am confused with Lagrange duality theorem. Let us consider the problem $$ \hat{\beta} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} \left[\sum_{i=1}^{n}(y_{i} - \beta_{i})^{2} + \lambda \sum_{i=...
user avatar
  • 73
3 votes
0 answers
34 views

Variable importance differs between lasso and random forest (R)

I am working with a dataset (~4000 subjects, 40 predictors, five continuous outcome variables) and I am interested in both feature selection and identifying the ordering of importance of various ...
user avatar
  • 41
1 vote
0 answers
28 views

Duality gap calculation in Scikit-learn implementation of Lasso

I am writing a custom variation of Lasso regression, using sklearn's Lasso implementation as a "source of inspiration". And I don't quite understand the very last line in the calculation of ...
user avatar
0 votes
0 answers
15 views

Elastic net grouping property in logistic regression

The grouping property of the elastic net is a well-known property. The elastic net groups highly correlated variables together in its coefficient estimates. In Theorem 1 of the elastic net paper (here)...
user avatar
  • 283
1 vote
0 answers
18 views

alternative solution to fussed lasso

The question is related to strange result from fused lasso estimator Let us consider fussed lasso estimator: $$ \hat{\beta}^{FL} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{...
user avatar
  • 73
1 vote
0 answers
25 views

R: What is the difference of the Lasso for variable selection between the packages glmnet and hdm

For my PhD I use a Lasso approach in R for variable selection. Now, I used the package glmnet and also hdm. What is the difference of the basic lasso estimator for logistic regression in these two ...
user avatar
4 votes
1 answer
66 views

strange result from fused lasso estimator

Let us consider the following estimator: $$ \hat{\beta}^{F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} (y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}|, $$ which ...
user avatar
  • 73
2 votes
1 answer
26 views

How to perform lasso on a wide matrix? [closed]

I have a Matrix with almost 1000 samples (rows) and for each of this I have gene expression data for more than 16000 genes. I was trying to perform lasso with the ...
user avatar
2 votes
2 answers
86 views

When do Lasso and linear regression have same solution?

When do Lasso and OLS give the same solution when applied independently from each other? But more importantly for me: If you first run Lasso and say X* are the features that Lasso has not shrunk to ...
user avatar
1 vote
1 answer
42 views

Lasso regression prediction on test set is predicting towards the mean of the train set?

I am using lasso regression to predict age (continuous data) from a set having 2112 numeric features (indepedent variable). The training dataset contains around 2773 participants. The mean of that ...
user avatar
  • 111
5 votes
1 answer
77 views

Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
user avatar
0 votes
0 answers
17 views

Replicating LDA and LDA with LASSO constraints in Matlab - how to calculate the objective function in LDA and how to code LASSO in Matlab?

I am trying to reproduce some examples from an article: DALASS: Variable selection in discriminant analysis via the LASSO from Trendafilov and Jolliffe (2007). I have two questions about two parts of ...
user avatar
  • 1
0 votes
0 answers
15 views

Choose between residual sum of squares (RSS) and comfounded RSS?

In every course I have taken, I was taught to use the residual sum of squares as (part of) the loss function in regressions, either in simple OLS, lasso or other linear regression methods. Recently I ...
user avatar
3 votes
0 answers
41 views

Ridge Regression Alpha/Lambda: Basic Characteristics?

I fear this is an ill-posed question that has been asked a million times, but what are the basic characteristics of the penalty multiplier (usually called $\lambda$ or $\alpha$) in Ridge Regression (...
user avatar
1 vote
1 answer
88 views

Best practice for Post-Double Selection LASSO (pdslasso)

I'd like to have a clearer idea of the optimal approach to the post-double selection LASSO (paper, webpage). Take data on an RCT with 2 treatment arm dummies $D_1, D_2$ and a potential driver of ...
user avatar
  • 15
0 votes
1 answer
55 views

Forcing covariates to always be part of a Lasso model

I want to use a Lasso to predict outcomes for different policy scenarios. At the optimal degree of regularization obtained by cross-validation, one important variable in whose impact I'm interested in ...
user avatar
  • 1
1 vote
0 answers
31 views

Lasso regression only predicts order of model correctly when poly() is set raw = TRUE?

I'm looking at exercise 8 in chapter 6 of Introduction to Statistical Learning and have noticed that the ability of glmnet() to correctly identify the non-zero ...
user avatar
  • 11
1 vote
1 answer
88 views

Linear Regression with Lasso Regularization by using scikitlearn and scipy.optimize

i am trying to apply lasso linear regression with both scikitlearn and scipy.optimize min method. However, i cannot reach same result. Code that i created with scipy.optimize can't shrink redundant ...
user avatar
1 vote
1 answer
57 views

Why OLS perform better than LASSO?

I am comparing OLS and LASSO regression for survey data. I have n>p, but I think my data is high-dimensional data as the p is 3000 and n is 48000. I am using k cross-validation. The results are ...
user avatar
  • 11
1 vote
1 answer
119 views

Questions related to survival analysis and Lasso Cox regression

I am doing several survival analyses on TCGA (The Cancer Genome Atlas) data. As this is my first time doing this kind of analysis, I have a question about it. To study the influence of the gene ...
user avatar
  • 13
1 vote
0 answers
78 views

Can I make Stata run lasso faster?

I am trying to run a lasso in Stata. I have 1.5 million observations and 1700 variables. Stata is running too slow. I am in 36th grid after 4 days. And get slower ever grid. I am using a 98GB Memory, ...
user avatar
  • 11
1 vote
1 answer
60 views

Why lasso cannot be arbitrarily applied?

Consider any log likelihood function $f(\theta|x)$ where $x$ is data. I can consider $f(\theta|x)+\lambda||\theta||_1$ where $||\theta||_1$ is the standard $L_1$ norm. It seems that I cannot apply ...
user avatar
  • 911
2 votes
0 answers
36 views

Is regularization in Keras equivalent to a standard Ridge or Lasso problem?

With the python package Keras, you can use $\ell_2$ or $\ell_1$ regularization but you have to use the option on each layer. But I definitely cannot tell if using ...
user avatar
0 votes
3 answers
46 views

Why does test MSE always decrease with increasing training size (and decreasing test size)?

Context: I am trying to find the best predictive model for a dataset with 1000 observations. The problem is I am not sure what the best training and test size should be. So what I did was that I ran ...
user avatar
0 votes
0 answers
17 views

Selecting characteristics for prediction of stock returns (Adaptive Group Lasso in R)

In attempt to find out what drives the predictive power and not the explanatory power of cross sections of expected return. We attempt to split the characteristics of stocks in quadratic splines, ...
user avatar
  • 1
0 votes
0 answers
48 views

Use LASSO & bootstrapping for inference stats - no machine learning

I have a simple question: it is valid to use a LASSO model for variable selection in a small dataset? I won't do any machine learning. The goal is to use LASSO for variable selection instead of ...
user avatar
  • 1
0 votes
0 answers
18 views

Validating features that were already selected by train-test of a LASSO model

I have 50 tagged samples. I have selected features out of lots of possible features (10,000 maybe) I would like to test their ability to predict the tags. I tried to train lasso/ridge models on a ...
user avatar
2 votes
0 answers
41 views

Min-max scaling vs standardizing in LASSO

I know that it is recommended to have features on the same scale for LASSO, such that the scale does not affect the penalty. However, does it matter whether or not features are scaled using $\frac{x-\...
user avatar
  • 21
4 votes
1 answer
158 views

Adaptive LASSO, confidence interval and sample size

I have almost no experience with math or stat, but I am trying to run an Adaptive LASSO on a continuous outcome with around 200 cases and a list of around 19 variables. Some of these variables have 3 ...
user avatar
  • 43
0 votes
0 answers
64 views

Does it make sense that the loss function for traning and evaluaton is different?

Huber loss function is widely used, because it combines the good properties of squared and absolute losses. Therefore, when I apply the penalized regressions, i.e. LASSO, Elastic net and Ridge, to ...
user avatar
0 votes
0 answers
14 views

Formal way to test what kind of differencing is necessary?

I'm working on a project that concerns time series data for South-Africa. My series has 34 explanatory variables and only (!) 30 yearly observations. The analysis is meant to be high-dimensional, ...
user avatar
  • 1
0 votes
1 answer
43 views

How to extract MSEP or RMSEP from lassoCV?

I'm doing lasso and ridge regression in R with the package chemometrics. With ridgeCV it is easy to extract the SEP and MSEP values by ...
user avatar
  • 21
1 vote
1 answer
84 views

LASSO vs. Standard Variable Selection via p-value

How can I reason about compare/contrast variable selection between LASSO running a standard multi-variate regression and setting betas to zero if the p-value is > 0.05 ?
user avatar
  • 421
1 vote
2 answers
41 views

Introduction to Statistical Learning Eq. 6.12 and 6.13

Can someone please explain me how the optimization of 6.12 leads to 6.14 and the optimization of 6.13 leads to 6.15?
user avatar
  • 93
0 votes
0 answers
33 views

Help setting up groups in glmmlasso in R

I am trying to use glmmlasso package and perform group lasso. Out of the box each categorical variable is treated separately and if you want lasso to treat them as a group you need to pass in that ...
user avatar
0 votes
0 answers
10 views

Different Time Series CVs?

I'm writing a synthetic control algorithm which uses rolling-origin cross validation. Upon reading my paper, others have suggested I use "forward cv" and another paper I read seems to refer ...
user avatar
0 votes
0 answers
47 views

Regularization Terms in MLE

Can you add regularization terms to any likelihood function you're trying to maximize? (e.g. L2/Tikhonov, Lasso terms) I'm used to seeing this done with simple quadratic loss functions (e.g. for ...
user avatar
  • 117
1 vote
0 answers
34 views

Partial derivative of a Group Lasso

I am looking at the gradient descent method for group lasso questions. Here's what I am currently stuck at. Given the quadratic form of the objective function: $$ f(x) = \frac{1}{2} x^T V x - m^T x + \...
user avatar
4 votes
2 answers
255 views

Relationship between laplace and l1 regularization

It is well known that an L1 regularized linear regression is equivalent to a regression with a Laplace prior on the distribution of the coefficients. This is explained here: https://bjlkeng.github.io/...
user avatar
0 votes
1 answer
29 views

gamlasso from plsmselect package returns null for lasso

I was fitting a generalized additive model (GAM) with LASSO penalty to my data using gamlasso, as in the following (here I use simulated data, but the idea is ...
user avatar

1
2 3 4 5
27