# How can (L1 / L2) regularization be equivalent to using a prior when priors can't be changed?

I understand the argument for how training with an L1/L2 regularizer is the same thing as finding the MAP estimate when the prior is Gaussian/Laplace. But there's a crucial difference. In Bayes' theorem, the prior must not be influenced by the data, while in practice ML people tend to tune the regularizer to maximize the validation score. This seems to contradict the Bayesian interpretation. In fact, it sounds closer to "Empirical Bayes". How would someone in both the ML and Bayesian communities respond to this?

• I think when people mention this equivalence, they assume a fixed regularization parameter. Tuning the regularization parameter using validation data is similar to "tuning" your choice of prior using the validation data Commented Apr 30, 2018 at 16:30
• Possible duplicate of How is empirical bayes valid?
– jld
Commented Apr 30, 2018 at 16:56

If you have a hyperprior $P(\alpha)$ (where $\alpha$ is the regularization parameter), then $P(\alpha|x) = \frac{P(x|\alpha)P(\alpha)}{P(x)}$. If we make the assumption that $P(\alpha)$ is quite wide and nearly constant within a wide range of values, then $P(\alpha|x) \propto P(x|\alpha)$, so we can select our regularization based on whatever works best on the validation set.