# Questions tagged [penalized]

Methods of modifying objective functions to control the solutions of optimization problems.

130 questions
Filter by
Sorted by
Tagged with
12 views

### Negative Binomial Regression Shrinkage Priors P greater than N case

Can someone suggest some literature review paper or any literature regarding Negative Binomial Regression with Shrinkage priors particularly P>N case? P: no. of variables, N: no. of observations in a ...
8 views

### What’s the difference between PQL regression and ridge or lasso or elastic net?

What’s the difference between the penalized methods, glmmPQL and elastic or ridge or elastic net?
42 views

### Signs of the lasso solution

Let us consider the following lasso estimator: $$\hat{\beta}_{L} = \arg\min \, \frac{1}{n}\sum_{i}^{n}||y_{i} - \textbf{x}_{i}\beta||_{2}^{2} + \frac{\lambda_{n}}{n}\sum_{j=1}^{p}|\beta_{j}|$$ ...
37 views

### Lasso and its dual: rates of regularisations

Let us consider the following lasso estimator: $$\hat{\beta}_{L} = \arg\min \, \frac{1}{n}\sum_{i}^{n}||y_{i} - \textbf{x}_{i}\beta||_{2}^{2} + \frac{\lambda_{n}}{n}\sum_{j=1}^{p}|\beta_{j}|$$ For ...
16 views

### Interpretability of the lasso coefficients

I have implemented the lasso method to a data set with 1000+ variables. I have reported the MSE value on the test set and the number of nonzero variables. I need to report the interpretability of the ...
105 views

### cross validation in lasso problem

Let us consider the following lasso estimator: $$\hat{\beta}_{L} = \arg\min \, \frac{1}{n}\sum_{i}^{n}||y_{i} - \textbf{x}_{i}\beta||_{2}^{2} + \frac{\lambda_{n}}{n}\sum_{j=1}^{p}|\beta_{j}|$$ and ...
53 views

### Howe to perform ridge regression only on a subset of the variables

I am trying to code some algorithm that performs ridge-regression with penalty parameter $\lambda$ on all features except for a specific subset. Let $\mathbf{X}$ be the $n \times p$ matrix for $n$ ...
8 views

### Is the SCAD a sort of BLUE estimator in high dimension?

Given a response vector $y$ and a data matrix $X$, the OLS estimator can be obtained by solving the minimization problem: $$min_{\beta}\left\{\|y-X\beta \|_2^2\right\}$$ Under the OLS hypothesis, ...
88 views

11 views

### Non-differentiable regression penalties having non-zero probability of inducing sparsity

I was wondering if anyone has any analytic insight into why non-differentiable penalties can set coefficients to zero, and if there is a more relaxed alternative to this definition regarding the ...
34 views

### Lasso: more penalization with more data?

I am currently doing a backtest of a financial data set with an expanding window. For this, I estimate a Lasso model each month. Hence, each month that I estimate the model, I will have more data. ...
31 views

### How am I supposed to understand the penalty term to penalize lack of smoothness?

The penalty term is written as $\lambda\int f''(t)^2 dt$. I am told that if $f$ is wiggly, then $f''$ is big, so this term penalizes the lack of smoothness of $f$. Uh, okay, that makes sense ... but ...
74 views

### Regression penalisation to keep correlated variables

I am an avid user of logistic regression and more specifically of penalised logistic regression. With standard penalisation (Ridge, Lasso, Elasticnet) the goal is to avoid overfitting and translates ...
25 views

### robust regression in glmnet?

I have data with many errors in the input. However, I also have much more independent variables than samples. I have been using glmnet for penalized regression. Is there a good algorithm like glmnet ...
142 views

### Creating a risk score from Cox Regression

I have two datasets with palliative cancer patients including 106 and 60 patients, respectively. I have biomarkers of inflammation and coagulation, as well as clinical characteristics for all patients....
79 views

### Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation

In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R. Although I have read many papers on the Firth ...
8 views

### Appropriate machine learning technique for spectral data and low-frequency feedback

I have a performance measure and a data source that basically supplies a complex and varying waveform. I would like to apply some unstructured learning technique to try and find a pattern in the ...
43 views

### Elastic net regression with uneven penalties for predictors

For a regression model where you are certain that y that depends on some predictors but are agnostic about whether some other predictors should enter, how should you incorporate this prior information?...
35 views

### Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
13 views

### In case of elastic-net regression, what is the updating equation of gamma0

In case of elastic-net regression, what is the updating equation of gamma0 I have a proble with dealing elastic-net regression for gamma and beta, ( argmin 1/2nSummation(yi-gamma-bxi)^2 + lambda(1-...
22 views

### Link between norm of weights/coefficients and smoothness

We often avoid overfitting by penalizing the norm of the weights/coefficients (in a classic Ridge or Lasso regression). I understand that we want smooth functions as they will be more likely to ...
91 views

### Why doesn't penalized cubic regression reduce the number of knots in a GAM?

As far as I understand, cubic regression penalization prevents overfitting by reducing the number of knots by penalizing wiggliness. The supplied parameter k serves ...
100 views

### LASSO regression: which method is better for selecting $\lambda$ in this case?

I am currently working on a method for adaptive knot placement in Spline regression. Following Osborne et.al. (1998), Yuan et.al. (2014) I am interested in using LASSO regression to select a subset of ...
116 views

### A reward becomes a penalty if

I am working to build a reinforcement agent with DQN. The agent would be able to place buy and sell orders for a day trading purpose. I am facing a little problem with that project. The question is "...
111 views

### why does lasso select at most n predictors?

From the seminal paper on elastic net regularization from Zou and Hastie 2005, I read ...
69 views

### Removing the intercept term for penalized logistic regression

I am working on lasso logistic regression and am trying to remove the intercept term from the penalty function. I have tried to use the mean centering theory but for logistic regression it can not be ...
92 views

### Large value of $X\beta$ in logistic regression?

In logistic regression, the probability is obtained from $$Pr = \frac{\exp(X\beta)}{1 + \exp(X\beta)} ~~~~ (1)$$ From the plot below, it is obvious that if $X\beta$ > 10, the probability approaches ...
71 views

### Confidence limits for constrained penalized log likelihood model

I am estimating parameter $\beta$ as: \begin{align} \hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
53 views

I'm currently working trough some examples which should finally end in a DQN Reinforcement Learning for the CartPole example in the openAI-Gym. Copied some code from GitHub which isn't deep yet: <...
390 views

### Calibration of penalized (LASSO or ELasticNet) logistic regression models

I would be very grateful for any help me with the following general query regarding calibration of penalized models with a binary outcome. I would like my prediction model to be calibrated (mean ...
121 views

### Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
316 views

### Elastic net visualization [closed]

Is there a way to display in a graph the elastic net (or penalized regression in general) results? Specifically, how can I render the coefficients of the variables?
764 views

### Does Regularized Logistic Regression Produce Calibrated Results?

It has been asked and addressed here that logistic regression modelling is calibrated already and there is no need for calibration of it. To me it seems the argument provided there does not follow ...
511 views

### Sample size calculation for elastic net regression

I am using elastic net regression to investigate the effect of preditors on the response variable while accounting for multicollinearity among the predictors. But I wish to perform a sample size ...
43 views

### How to prove oracle properties in Fan and Li (2001) paper

I am studying Fan and Li's 2001 paper "Variable selection via nonconcave penalized Likelihood andits oracle properties" but I am having troubles understanding Theorem 1 proof (page 1359). I follow the ...
568 views

### Interpreting glmnet Lasso coefficients on dummy variables (multiple levels) [duplicate]

I am trying to apply glmnet's lasso to a set of features in which there are multiple categorical variables with multiple levels. My intention is to let lasso reduce ...
78 views

### Using covariates from penalized regression model in unpenalized model

The good news where I am is that researchers are doing less stepwise covariate selection now that I've introduced penalized regression. The bad news is that researchers want to use elastic-net ...
196 views

### In LASSO, does it make sense to choose lambda based on the mean error associated with different lambda values, over multiple cross-validations?

I am running a LASSO regression, but am put off by the different values of lambda each time I run the cross-validation. Does it make sense to run a cross validation multiple times, take the mean error ...
2k views

### Why is R Squared not a good measure for regressions fit using LASSO?

I have read in several places that R Squared is not an ideal measure when a model is fit using LASSO. However, I'm not clear on exactly why that is. In addition, could you recommend the best ...
123 views

### Equivalent of using a Poisson prior in terms of a penalized regression?

I know that most penalized regressions have also a Bayesian interpretation, e.g. ridge least squares regression corresponds to the MAP estimate obtained under a Gaussian prior in a Bayesian regression,...
138 views

### How do you explain many optimal models in penalized Logistic regression?

I am building penalized logistic regression using Lasso and Ridge methods. I know that the best model chosen by the program is which has alpha = 1 and lambda = 0.06....
107 views

### “beta_given” column in the h2o.glm beta_constraints

What does the "beta_given" column do in the h2o.glm beta_constraints parameter? h2o is an open source library for machine learning algorithms. There are several online examples on how to install the ...
I have a null model $M_0$ with parameter $\theta$ and a more general model $M_1$ with parameters $(\theta,\alpha)$. I know that $M_0=M_1$ for $\alpha=0$. I am estimating $M_1$ using a penalized ...
### How is the minimum $\lambda$ computed in group LASSO?
The LASSO problem works by minimizing $$\min_\beta (\frac{1}{2}\left\rVert y-X\beta\right\rVert^2_2+\lambda\left\rVert\beta\right\rVert_1)$$ Here in this webpage I found that the minimal value of ...