# Tagged Questions

A regularization method for regression models that shrinks coefficients towards zero.

11 views

### applying Posterior predictive distribution on the data from which the coefficient of regression were estimated [on hold]

I am new to linear regression modelling. For the given linear model $Y_{current} = \mu+ \beta X_{current} + \varepsilon \sim N(0|\sigma^2)$ Generally, in regression analysis we estimate, coefficient ...
68 views

### How freqently are the information conditons for proper Akaike information criterion application actually met?

Akaike information criterion (AIC) is limited to goodness-of-fit for assumed distributions. That is, for an assumed distribution (especially homoscedastic) one can assume a maximum likelihood ...
192 views

### Ridge/Lasso Lambda greater than 1

I ran Ridge and Lasso regressions using an algorithm to automatically find the optimum lambda. However, the algorithm couldn't find an optimum lambda between 0 and 1. In some cases I could find ...
29 views

### Understanding Kernel Ridge Regression and How It Works (and Implementing it in R)

I am trying to understand how KRR works for drug-protein-interaction and many aspects of it seem very confusing. Supposing I have a data set as follows of Drug-Protein interactions; values show how ...
98 views

### How is L1 regulaziation derived?

I understand the basic idea of regularization. I am very curious to know the derivations behind it so that I get the complete picture. I though a good place to start leraning about regularization is ...
12 views

### OLS or Ridge in Multicollinearity data

I am new to stats and linear regression. I just want to understand the exact scenario and usage between Ridge and OLS. Here is the data sample i have been using. In this both Weight and BSA are ...
55 views

### Lasso and Ridge tuning parameter scope

In ridge and lasso linear regression, an important step is to choose the tuning parameter lambda, often I use grid search on log scale from -6->4, it works well on ridge, but on lasso, should I take ...
12 views

### fast way to train a classifier on different but overlapping features

I am training a linear classifier repeatedly on different set of overlapping features. I have a 3D grid of features, each time features from a small sphere from a grid are used to train a classifier, ...
46 views

### Can L1 linear regression perform worse than vanilla linear regression on fewer features?

I have a data set with 2 features and I'm trying to predict one real-valued variable. I use linear regression and I measure the error using 10-fold CV and absolute mean error as a metric. I noticed ...
15 views

### Generalized Tikhonov regularization in glmnet?

Is it possible to do Generalized Tikhonov regularization https://en.wikipedia.org/wiki/Tikhonov_regularization#Generalized_Tikhonov_regularization with glmnet? This seems like straightforward and ...
36 views

### Defining lambda value for ridge regression

I try to understand ridge regression. I think the most important point in it is defining lambda value. I researched some R codes but I didn't understand how to define it. In the example here, in the ...
40 views

### Interpreting Special Case for Ridge Regression and the Lasso

The below text is from Statistical Learning Page no.225 Consider a case with $n = p$, and $\mathbf{X}$ a diagonal matrix with 1’s on the diagonal and 0’s in all oﬀ-diagonal elements. To simplify ...
28 views

### Tikhonov regularization in the context of deconvolution

I came across "Tikhonov regularization" and I have bare knowledge on it. It seems that it is a type of regularization that is important for deconvolution. Are there any good resources and examples? ...
37 views

### Predicting of revenue. Penalized regression (Ridge regression)

I have data of sales. I've selected one point of sales to check a possibility of predicting revenue using regression method (I don't know what can I use in this task). First of all I've tried to find ...
104 views

### double feature value in ridge regression, coefficients change?

In ridge regression using unnormalized features, if you double the value of a given feature A (i.e., a specific column of the feature matrix), what happens to the estimated coefficients for every ...
24 views

### Replacing ridge regression with Bayesian MCMC

I have a ridge regression model $y = \beta_1 x_1 + \beta_2 x_2 + ...$ The $x$s are highly collinear but are all physically relevant, hence use of ridge regression. And am considering replacing ...
59 views

### Can I implement ridge regression in terms of OLS regression?

Can I implement ridge regression in terms of OLS regression? Is it even possible? I am interested because scikit-learn supports non-negative least squares (NNLS), ...
73 views

### How to perform non-negative ridge regression?

How to perform non-negative ridge regression? Non-negative lasso is available in scikit-learn, but for ridge, I cannot enforce non-negativity of betas, and indeed, ...
20 views

### Difference of feature importance from Random Forest and Regularized Logistic Regression

I have 13 features in a classification task and I use Random Forest, L1 logistic regression and L2 logistic regression for as separate classifiers and would like to compare their performance. Although ...
5 views

### Bounding the loss for kernel regularization algorithms

Some learning bounds depend on a quantity $M$ that is the maximum loss a learning algorithm can have: so $L(h(x),y) \leq M$, where $h(x)$ is the prediction of the model, and $y$ is the label, and $L$ ...
9 views

22 views

### Why is intercept excluded in ridge regression? [duplicate]

Why don't we include the intercept term in ridge regression, i.e exclude $w_0$ term in calculating the coefficients? Why is it so?
42 views

### The scaling problem of ridge regression

I have been confused with the scaling of ridge regression input for a long time. There are several sources about how to do the scaling: Just do the centering to input(From "The Elements of ...
60 views

### Decompose ridge regression bias error into model bias and estimation bias

How can I show that the in-sample bias error in Ridge regression can be decomposed into model bias plus estimation bias? I.e., if $Avg$ takes the average over all the input variables $x$ in the ...
162 views

### Computing cross-validated $R^2$ from mean cross-validation error

I am currently using cv.glmnet in R. I would like to compute both a training $R^2$ and a cross-validated $R^2$. R gives mean cross-validated error and for the ...
52 views

### Meaning and significance testing of coefficients in lasso/ridge regression

Can somebody explain the importance of significance testing in ridge/lasso regression? Is it necessary to do it? And, how can we interpret the coefficients of ridge regression which are penalized?
96 views

### Elastic net produces complex output with too many non-zero coefficients

I have run 3-fold cross-validation for elastic net using elasticnet R function on ~200 observations and using 80 variables (and there will be some more). Both ...
322 views

### Why is ridge regression giving different results in Matlab and Python?

Why is the output from Matlab and Python vary for ridge regression? I use the ridge command in Matlab and scikit-learn in Python ...
151 views

### Can ridge regression be used in the presence of categorical predictors?

I have a regression problem and I am thinking of using ridge regression. One of the predictors is subject's gender, which is a categorical variable. How to take care of this variable for ridge ...
52 views

### First step of Kernel Ridge Regression?

I want to implement kernel ridge regression (KRR) using a polynomial kernel as a function that takes the training objects, training labels and test objects as arguments, and outputs the vector of ...
10 views

### First step of Kernal Ridge Regression? [duplicate]

I want to implement kernel ridge regression (KRR) using a polynomial kernel as a function that takes the training objects, training labels and test objects as arguments, and outputs the vector of ...
66 views

### Do methods exist other than Ridge Regression and Y ~ X + 0 to prevent OLS from dropping variables?

Goal is to evaluate chess players using a novel analysis system I'm been working on -- not all wins are created equal, finding the only move in razor sharp positions is better than finding the best ...
584 views

### What is elastic net regularization, and how does it solve the drawbacks of Ridge (L2) and Lasso (L1)?

Is elastic net regularization always preferred to Lasso & Ridge since it seems to solve the drawbacks of these methods? What is the intuition and what is the math behind elastic net?
606 views

### When will L1 regularization work better than L2 and vice versa?

Note: I know that L1 has feature selection property. I am trying to understand which one to choose when feature selection is completely irrelevant. How to decide which regularization (L1 or L2) to ...
14 views

### How to model the discrepancy between test result and prediction code, and find the main sources of discrepancy

I have data about some tested machines, stored as a database with $\approx$ 300 observations, 40 predictors, 2 outcomes. The two outcomes (responses) are the result of the machine test for that ...
15 views

### Ridge regression on centred or uncentered data? [duplicate]

Today is the first day that I have heard about ridge regression, so please do not judge harshly. I have learned that you use ridge regression when there is a case for multicollinearity of your ...