Timeline for If shrinkage is applied in a clever way, does it always work better for more efficient estimators?
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 21, 2017 at 13:46 | comment | added | TPArrow | Let us continue this discussion in chat. | |
| Jun 21, 2017 at 13:38 | comment | added | user795305 | In particular, notice that $\hat\beta_1$ and $\hat\beta_2$ are given, and we seek to apply some shrinkage technique to them. | |
| Jun 21, 2017 at 13:38 | comment | added | TPArrow | @Ben thanks. I will do it surely. However, I do not think that it affects the actual answer. | |
| Jun 21, 2017 at 13:26 | comment | added | user795305 | Shrinkage can be both inline and as a post-processing. The examples you mentioned in your response are about "inline shrinkage", while the question asks about "post processing shrinkage". Notice that the question gives two estimators $\hat\beta_1$ and $\hat\beta_2$, then asks for a shrinkage technique to apply to $\hat\beta_1$ or $\hat\beta_2$. I think it might be worthwhile to reread the question in light of this. | |
| Jun 21, 2017 at 11:59 | comment | added | TPArrow | thanks @Ben, I feel we do not have a consensus in the definition of shrinkage. You take it like a post-process but me as an inline processing. I think we are both right since the question is not taking the type of shrinkage into account. PS: I guess what you mean from shrinkage is like hard-thresholding. | |
| Jun 20, 2017 at 15:21 | comment | added | user795305 | It's still not really clear to me. Are you proposing that we take the initial estimates $\hat\beta_1$ and $\hat\beta_2$ and then evaluate the $\ell_p$ proximal operator of them, so that the new estimates are $\hat\alpha^p_j = \arg\min_\alpha \|\alpha-\hat\beta_j\|_2^2 + \lambda \|\alpha\|_p$, for $j \in \{1,2\}$? If so, could you provide a proof (or some other argument) for your claims regarding MSE improvement? I tried to emphasize earlier that the question is asking about post-processing estimators--what exactly are your estimate for $p=2,3$ post processing? | |
| Jun 20, 2017 at 14:51 | comment | added | TPArrow | @Ben Thanks. the question is about shrinkage and I tried to take a simple example where applies shrinkage by imposing $l_p$ norm penalty on the estimator. I see it quite related. PS: LASSO ($l_1$ norm penalized likelihood) stands for Least Absolute Shrinkage and Selection Operator | |
| Jun 20, 2017 at 14:21 | comment | added | user795305 | it's not clear to me what you mean by take $p \to 1$. Given two estimators (say, from having $p=3$ and $p=2$ in $\ell_p$ regularization of least squares, like you discuss in your response), the question asks about ways to post-process these estimators (via, say, shrinkage). Specifically, it asks if there exists methods that can produce similar improvement (in terms of MSE) across consistent and asymptotically normal estimators. It's not clear to me what your answer is supposed to convey related to this. | |
| Jun 20, 2017 at 9:16 | history | answered | TPArrow | CC BY-SA 3.0 |