- Gaussian prior on weights of a GLM lead to Ridge / $\ell_2$ squared regularization.
- Laplace prior leads to $\ell_1$ regularization
Question
What prior would lead to $\ell_\infty$ regularization ?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityWhat prior would lead to $\ell_\infty$ regularization ?
Thinking of similar examples, the reason behind the correspondence between these priors and norms is the exponent part. $$f(\beta)\propto \exp(-\lambda||\beta||_1)\rightarrow \ell_1$$ $$f(\beta)\propto \exp(-\lambda||\beta||^2_2)\rightarrow \ell_2 \text{ squared}$$ So, if we have a prior in the form below, it will correspond to $\ell_\infty$ because the negative log likelihood will directly include the term $\lambda||\beta||_{\infty}$ as a summand. $$f(\beta)\propto \exp(-\lambda||\beta||_\infty)\rightarrow \ell_\infty$$
I don't know if this PDF has a name, but it is a proper one ($n$ is the dimension): $$f(\beta)=\frac{\lambda^n}{2^nn!}\exp(-\lambda||\beta||_\infty)$$