Timeline for Using regularization when doing statistical inference
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 19, 2020 at 16:03 | comment | added | Melkor.cz | If L1 and L2 penalties are equivalent to certain Laplace and Gaussian priors, how can one determine the regularization parameter lambda required for given Laplace or Gaussian distribution parameters? Say I have multiple regression with categorical predictors and want to impose prior ~N(0, 1), how do I determine the lambda? | |
| Jul 19, 2017 at 6:24 | comment | added | user795305 | (+1) Good answer! However, I think it's may be worthwhile to clarify the sentence " But most Bayesian don't use MAP estimation, they sample from the posterior distribution using MCMC algorithms!" It seems like you're trying to say that most Bayesians use the full posterior in choosing their estimator. To see the problem, note that an estimate for the MAP can be made from the sample for the posterior distribution. | |
| Jul 14, 2017 at 23:08 | comment | added | user3903581 | A word of caution: The prior + MCMC approach will only give valid results if the posteriors for all potential coefficients are examined and reported. Otherwise, we are in a selective inference setting and most naive inference methodologies will be invalid. | |
| Jul 14, 2017 at 22:53 | comment | added | Cliff AB | @Scortchi: that's a very good point: using cross-validation to choose penalties takes you well out of the classical Bayesian framework (as far as I know). Building a model with CV to choose regularization parameters would not fall coincide with this answer, but using regularization with fixed penalties, chosen based on expert information would. | |
| Jul 14, 2017 at 21:53 | comment | added | Scortchi♦ | (+1) But if I've used those priors only because they give good predictions - indeed I may well have tuned them for that purpose - then what am I to make of the MAP estimates or posterior distributions? (Of course if I elicited the priors to represent knowledge about the parameters before seeing the data I know exactly what to make of them.) | |
| Jul 14, 2017 at 20:59 | history | answered | Cliff AB | CC BY-SA 3.0 |