# Bayesian Regression: Loss Function Explained

Considering a simple linear regression model e.g. $$y_i = \alpha + \beta x_i + \epsilon$$ , in probabilistic terms: $$\mu_i = \alpha + \beta x_i$$ $$y_i \sim \mathcal{N}(\mu_i, \sigma)$$

We assume prior distributions on the parameters $$\alpha, \beta, \sigma$$ and apply Bayes theorem (posterior $$\propto$$ likelihood x prior) i.e.:

$$f(\alpha, \beta, \sigma | Y, X) \propto \prod_{i=1}^{n} \mathcal{N}(y_i; \alpha + \beta x_i, \sigma) f_\alpha(\alpha)f_\beta(\beta) f_\sigma (\sigma)$$

With priors and data, we then perform Markov Chain Monte Carlo method for sampling from this posterior distribution.

My question is:

• How do loss functions or different loss functions fit into this picture? How do loss functions come into the next stage of taking a decision based on the posterior distrubtion?
• Loss functions come into the next stage, where you take a decision based on the posterior distribution, using the loss function to help decide which decision is optimal. Nov 23, 2021 at 9:08
• That is useful thanks @Henry Nov 23, 2021 at 10:05

Same applies to using regularization. In Bayesian setting, regularization is dealt by choosing appropriate priors. For example, $$\ell_2$$ penalty is equivalent to using Gaussian priors for the parameters.