Timeline for Should MLE estimation always be using penalizers?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 17, 2019 at 6:24 | answer | added | Stephan Kolassa | timeline score: 2 | |
| Apr 17, 2019 at 1:43 | comment | added | Cam.Davidson.Pilon | I understand your point, but one can still reasonably guess an approx value. However, I think your suggestion of CV is totally valid! Let's use that. | |
| Apr 17, 2019 at 0:57 | comment | added | jbowman | The magnitude of the penalty term has to be tied to the problem scale; if I divide the target variable by 100, all my coefficients in, say, a linear regression, will shrink by 100 as well, so the penalty had better too. That's why you can't come up with off-the-shelf penalties. Also, consider a Lasso regression with 100+ regressors; guessing a good penalty for something like that isn't likely to be much more feasible than guessing the coefficients themselves. Why guess at what penalty is when you can use cross-validation or a reasonable approximation thereto to select a good value? | |
| Apr 16, 2019 at 22:47 | history | edited | Cam.Davidson.Pilon | CC BY-SA 4.0 |
added 24 characters in body
|
| Apr 16, 2019 at 22:45 | comment | added | Cam.Davidson.Pilon | I must have not been clear, but I meant that adding a penalizer could lead to broken/misrepresentative CIs. A sensible value for a penalizer: 1e-5 - not too big to strongly influence results, but also big enough to discourage unrealistic effect sizes. (Also, Bayesian practitioners come up with sensible prior values all the time, do you also think they are walking on thin ice?) | |
| Apr 16, 2019 at 22:26 | comment | added | jbowman | With respect to the last point, if I observe a bunch of failure rate data and all my time to fails are less than two years, I don't need a penalizer to tell the MLE estimation routine that 15 years MTBF is not a likely value. You seem to be way overstating the benefits of penalization... why do you think confidence intervals for non-penalized estimates might be "drastically broken" but wouldn't be if you added a penalty function? Why do you think a practitioner would have any idea what a sensible value for a penalty term would be? I sure wouldn't, and I've been doing this for years... | |
| Apr 16, 2019 at 21:51 | comment | added | jbowman | Point #2 is incorrect, consider for example the MLE of the mean of a Normal distribution, which is already minimum variance in the univariate case. | |
| Apr 16, 2019 at 21:32 | history | asked | Cam.Davidson.Pilon | CC BY-SA 4.0 |