# Questions tagged [regularization]

Inclusion of additional constraints (typically a penalty for complexity) in the model fitting process. Used to prevent overfitting / enhance predictive accuracy.

1,298 questions
Filter by
Sorted by
Tagged with
1 vote
24 views

### Stepwise model selection by AIC

I am learning about performing stepwise model selection by AIC and having some questions: What is the regularization parameter for step-AIC? In what way is forward step-AIC an evolution of univariate ...
• 41
83 views

### In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
• 31
1 vote
15 views

### Do I need to normalize data before applying L1, L2 norm in ANN

I wish to train the ANN and use regularizers to avoid overfitting. I need some suggestions, is it mandatory to normalize the data before using L1, L2 regularizers. I would highly appreciate if you can ...
• 141
11 views

### Apply Shrinkage Coefficient to Exponentially Weighted Moving Covariance Matrices

Title says it all. I'm working on doing volatility forecasting and like the approach of exponentially weighted moving covariance matrices, but I also know that applying shrinkage coefficients further ...
10 views

### The use of the index argument in glmmlasso in R with interaction terms

I am using the glmmLasso package in R for variable selection with repeated measurements. I have 14 variables + interactions of each of these 14 variables with age that I want to use the selection for. ...
11 views

### Shrinkage / L1 regularization as a loss term versus a constraint (post-process step) with momentum optimizers

I have a complex model with very non-linear operations (divisions, exponentials, matrix inversions, square roots, Cholesky decompositions, etc...) for which I want to optimize the parameters. However, ...
17 views

• 436
12 views

### Would l-1 regularization with kernel trick induce sparsity on feature map's features?

Would l-1 regularization with kernel trick induce sparsity on the infinite dimensional feature map's features in the case of gaussian kernel?
• 224
8 views

### Would logistic regression/support vector-machine with l-2 regularization and early stopping regularization cause underfitting?

Would early stopping regularization combined with l-2 regularization or in logistic regression/support vector machine cause underfitting? Does a kernel-trick affect what combination of regularization ...
• 224
13 views

### Beta distribution equivalence with two redondant parameters [duplicate]

context In Factor graphs on discrete variables, the parameters are contained in factors associated each with a subset of the random variables in the system. Each factor provides a different positive ...
2k views

### If I use a regularization (e.g. L2) can I not apply early stopping?

I've seen that early stopping is a form of regularization that limits the movement of the parameters $\theta$ in a similar way that L2 Regularization penalizes the movement of $\theta$ to be closer to ...
• 287
28 views

### Is regularization in Keras equivalent to a standard Ridge or Lasso problem?

With the python package Keras, you can use $\ell_2$ or $\ell_1$ regularization but you have to use the option on each layer. But I definitely cannot tell if using ...
46 views

### Deciding between the L1 and L2 penalty for a Sklearn Logistic Classifier

I have a classification problem with the following example independent features: recommendations comment_count comment. 0.663 . 0.382 'yes', 'trump' The dependent variable is whether the comment is ...
14 views

### Regressor-based L2 penalty [duplicate]

I'm working on a multiple regression problem where I have reasons to believe some (if not all) of the regressors have been cherry picked/data mined to a varying degree. My hypotheses are that there's ...
• 749
25 views

19 views

### Bayesian Approach for Underdetermined Datasets

If Bayesian Linear Regression with Gaussian prior produces L2 norm and Laplacian Prior produces L1 norm, is it fair to say that handling of underdetermined data sets (where number of columns > ...
42 views

### Can one use NRI and IDI in regularized cox-regression?

I have a dataset with 1500 patients for which I want to predict the outcome of death. I wanted to utilize multivariate cox-regression in a model containing biomarkers and other covariates. I was told ...
• 37