Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

# Questions tagged [shrinkage]

Shrinkage in statistics is a form of regularization consisting in changing parameter estimates to be "smaller" (closer to zero), or, more generally, "closer to each other".

130 questions
Filter by
Sorted by
Tagged with
27 views

### Lasso: more penalization with more data?

I am currently doing a backtest of a financial data set with an expanding window. For this, I estimate a Lasso model each month. Hence, each month that I estimate the model, I will have more data. ...
41 views

### Deriving posterior mean with horseshoe prior

I want to decompose a matrix $S \in \mathbb{R}^{D \times D}$ as below $$S=vv^T$$ where $v_i\mid\lambda_i,\tau_i \sim N(0,\lambda^2_i\rho^2_i)$, $\lambda_i \sim Cauchy^+(0,1)$ i.e $v$ has horseshoe ...
25 views

### How to Create a Toy Example of the Curds and Whey Algorithm?

Why is my simulated example failing? I am trying to create a toy example of the Curds and Whey method for multivariate linear regression in python (An example in R would be very helpful as well). I ...
15 views

### How to get change of basis matrix for Canonical Correlation Analysis?

A bit of background: I am trying to create toy example of the Curds and Whey regression shrinkage algorithm in python. In a standard multivariate regression this algorithm uses canonical correlation ...
36 views

372 views

### Why does shrinkage really work, what's so special about 0?

There is already a post on this site talking about the same issue: Why does shrinkage work? But, even though the answers are popular, I don't believe the gist of the question is really addressed. It ...
43 views

### Calibrating LASSO prior (how to select the scale hyperparameter)?

I want to use a LASSO prior (Laplace prior) for a location parameter $\mu$ $$\pi(\mu \mid s) = \dfrac{1}{2s}\exp\left(-\frac{\vert \mu \vert}{s}\right).$$ However, I do not know to calibrate this ...
112 views

### James-Stein regularizing covariance like a mean

In James-Stein's estimator we have a $p$-dimensional random vector $X\sim N_{p}(\mu ,I)$ where $\mu \neq 0$ and the goal is to estimate the mean vector using the single ($n=1$) data vector $X$. The ...
14 views

### Inflation of effect size estimation among significant results?

Correction for multiple testing is often performed on the alpha value. However, multiple testing is, I think, likely to inflate also the estimated effect size of the most significant effects found. ...
27 views

### Prove that the direction in Least Angle Regression makes equal angle with all predictors [closed]

Least Angle Regression iteralively adds predictors according to the procedure described here : Writing by hand first steps in Least Angle Regression (LARS) We note $A_{k}$ the active set of variables ...
129 views

### Writing by hand first steps in Least Angle Regression (LARS)

How do we write the first steps of Least Angle Regression ? What is the rationale behind this method ? What limitations of other methods is it overcoming ? Why is it called Least Angle Regression ?
60 views

### What are some good examples of working through a multilevel model by hand?

I've been learning about multilevel models lately, and I understand the concept of shrinkage and partial pooling (I think), but I'm still confused to some extent on how partial pooling actually ...
242 views

### Why not use Ridge after Lasso vs relaxed Lasso

Has anyone ever applied a ridge regression on a model subset selected from a cross validated lasso? In other words, take a data set with p features and run lasso, grid searched to find optimal ...
32 views

### How to give a represantation to veriables from each group using LASSO

I'm trying to apply LASSO regression on my data set in order to choose the best variables. However, my variables (44 to be accurate) come from 7 different groups, is there any option to give a "...
68 views

74 views

### Choice of ridge parameter in constrained estimation

Apologies for a possible basic question. Suppose I have a model $$y=X\beta + \epsilon, \quad E[\epsilon|X]=0, \quad X \text{ is } n \times k$$ and, say, matrix $X'X$ is nearly singular and I have a ...
44 views

1k views

### In Ridge regression and LASSO, why smaller $\beta$ would be better?

Can anyone provide an intuitive view on why it is better to have smaller beta? For LASSO I can understand that, there is a feature selection component here. Less features make the model simpler and ...
In one of the slides from my class related to bayesian linear regression, I have the following scenario. Under g-prior, the shrinkage estimator induced by the prior is \hat{\beta_{\alpha}} = \alpha\...