# Tagged Questions

Is a form of regularization used in estimation of regression coefficients which shrinks coefficient estimates by penalizing their absolute value (i.e. the $L_1$ norm of the estimates). The LASSO is equivalent to the Bayesian estimation problem where iid standard Laplacian prior is used for the ...

14 views

### How to sample from this dirichlet distribution with an L1 prior?

I'd like to draw a sample from a distribution with p.d.f $$f(p,q,r,s) \sim \mathrm{e}^{-w(|p+q-r-s|+|p-q-r+s|+|p-q+r-s|)}p^aq^br^cs^d \mathbb{1}_{p+q+r+s=1}$$ $w > 0$ is a free parameter (which ...
177 views

### Interpretting LASSO variable trace plots

I am new to the glmnet package, and I am still unsure of how to interpret the results. Could anyone please help me read the following trace plot? The graph was ...
70 views

### How to interpret the lasso selection plot [duplicate]

I did lasso selection using lars::lars(), then I got this plot. I have no idea how to interpret it: Could anyone provide a brief explanation? Why does it plot ...
55 views

### LASSO with L1 loss function

I've been trying to figure out a way to perform LASSO with L1 loss function (instead of the L2 loss) but have been completely dumfounded as to how. I've attempted to use the flare package's ...
299 views

### When wouldn't I use LASSO for model selection?

Assume that you need to build a linear model to make predictions for new observations, and that there is uncertainty about which subset of variables should be included in the model. You are only ...
21 views

### Sample size for LASSO

What considerations should be made regarding sample size for using LASSO or elasticnet? We are going to gather expression data from 1700 genes and our response variable is multinomial (3 categories).
104 views

### What's the tractable data size for Sparse PCA or LASSO?

I'm up to perform certain kinds of sparse decomposition methods on my dataset. However, I'm not sure what's the tractable data size for the Sparse Decomposition methods? The dataset is 10^3 * 10^5 ...
124 views

### Issues with using glmnet package for matlab

I am trying to use glmnet-matlab pacakge for training my elastic net model on some huge data. My features are of size 13200 and I have around 6000 samples of these. I directly tried to use lassoglm in ...
46 views

### How is $\lambda$ tuning parameter in lasso logistic regression generated

I know glmnet(x,y) generates $\lambda$ but I am very curious to know the actual formula that is behind this, generating $\lambda$.
78 views

### What is the time complexity of Lasso regression

What is the asymptotic time complexity of Lasso regression as either the number of rows or columns grows?
36 views

265 views

### An example: Lasso regression using glmnet for binary outcome

I am having confusion and difficulties using glmnet with lasso where my outcome of interest is dichotomous. I have created a small bogus data frame below: ...
203 views

### What is deviance in lassoglm

I am trying to fit a lasso penalized logistic regression model to a certain data. I am using lassoglm for that in matlab. I use the following function [B,FitInfo] = ...
22 views

### Confusion related to derivation of soft thresholding function

I was going through this paper and reading some problem(lasso type) that was being solved using soft thresholding, but I didn't get how it was derived. Can anyone please provide some suggestions? I ...
50 views

### How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
1k views

### What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
31 views

### Network/structure learning

Given a data set $\mathbf{X}\in\mathbb{R}^{n\times p}$, where $n$ is the number of samples (observations) and $p$ is the number of features, I would like to know what kind of methods exist for ...
70 views

### Running regularized logistic regressions on very large datasets

I want to run a regularized logistic regression on a dataset with 25 million observations and about a 1000 mostly non-sparse columns with non-ignorable weights. My first choice would be BayesGLM, ...
42 views

### Correct estimation of arguments for glmmLasso function

I am using glmmLasso for variable selection. In my case, n is slightly less than p and ...
39 views

### References for Bayesian group-Lasso for probit/logit regression

Does anyone have a paper or other references on Bayesian group-Lasso for probit/logit model or GLM (generalized linear models) in general? I could not find any paper that explicitly deals with this.
56 views

### Cross validation for lasso logistic regression

I am writing a routine for logistic regression with lasso in matlab. So the problem is to minimize the negative log-likelihood function with the penalty term: ...
2k views

### Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
59 views

### Cross validation for lasso logistic regression

I am writing a routine for logistic regression with lasso in matlab. So the problem is to minimize the negative log-likelihood function with the penalty term ...
7k views

### How to estimate shrinkage parameter in Lasso or ridge regression?

I want to use Lasso or ridge regression for a model with more than 50,000 variables. I want do so using software package in R. How can I estimate the shrinkage parameter ($\lambda$)? Edits: Here is ...
101 views

### How big are regularization parameters values?

I wanted to know how big are the regularization parameter values for ridge or lasso. I have seen most of the places generally using values like 0.1 or 0.01 but in some of my experiments the cross ...
41 views

### LASSO method: prediction for multi-dimentional reponses

I have a feature matrix, that is 'X' 2000 (observation) x 200 (variable). I also have a response matrix, that is 'Y' 2000 (response) x 2 (variable). I would like to apply LASSO method to the data ...
30 views

### LASSO and parameter uncertainty

I am using Bayesian model averaging (BMA) as a method for variable-selection, because of the fact that it incorporates model uncertainty. However, in paraellel, I have also attempted to use LASSO as ...
451 views

### Stepwise regression vs. elastic net

I understand that Stepwise regression analysis has lots of limitations, including the assumption that the predictors are not highly correlated with each other. In fact, this limitation was the most ...
66 views

### whether to rescale indicator / binary / dummy predictors for LASSO

For the LASSO (and other model selecting procedures) it is crucial to rescale the predictors. The general recommendation I follow is simply to use a 0 mean, 1 standard deviation normalization for ...
176 views

### Lasso cross validation

I want to perform cross validation to find the regularization parameter for Lasso. I am using scikit-learn library in python. I first generate the dataset and then perform k-fold cross-validation. ...
53 views

### Applying LASSO with cross-sectional/time-series data

Even after reading the original paper on LASSO, I am still confused as to how to apply LASSO for variable selection when the response variable y is a matrix (and not a vector) of n dependent ...
112 views

### Confusion related to elastic net

I was reading this article related to elastic net. They say that they use elastic net because if we just use Lasso it tends to select only one predictor among the predictors that are highly ...