1
$\begingroup$

Question: What are the advantages and disadvantages of using a Laplace likelihood in regression instead of a Gaussian likelihood?

Details: I know that if the unknown regression coefficients have a Gaussian prior and the likelihood function is also Gaussian, then the MAP estimate solution will be the same as Ridge regression.

Furthermore, if the regression coefficients have a Laplace prior and the likelihood function is Gaussian, then the MAP estimate solution will be the same as Lasso. It is because the log of the posterior distribution $ p(\beta|\{x_i,y_i\}_{i=1}^n)$ over the coefficients can be computed as follows: \begin{align} \log p(\beta|\{x_i,y_i\}_{i=1}^n) ~ \varpropto ~ - c (\sum_{i} (y_i - \beta^T x_i)^2 + \lambda \sum_j |\beta_j|) \tag 1 \end{align}

where $c$ is some constant, $\beta$ is the $p$-dimensional regression coefficients, $\lambda$ is the regularization parameter, and $\{x_i,y_i\}_{i=1}^n$ are the centered $p$-dimensional input and response pairs.

As seen in (1), MAP estimate maximizes $\log p(\beta|\{x_i,y_i\}_{i=1}^n)$ or equivalently minimizes $(\sum_{i} (y_i - \beta^T x_i)^2 + \lambda \sum_j |\beta_j|)$ which is also the same as Lasso.

However, if we have a Laplace likelihood, (1) changes to \begin{align} \log p(\beta|\{x_i,y_i\}_{i=1}^n) ~ \varpropto ~ - b (\sum_{i} |y_i - \beta^T x_i| + \lambda \sum_j |\beta_j|) \tag 2 \end{align}

Now the MAP estimate minimizes $(\sum_{i} |y_i - \beta^T x_i| + \lambda \sum_j |\beta_j|)$ which is neither the ridge regression or Lasso. So, what are the advantages of using (2) instead of (1)? And yet another question, how we can compute the MAP estimate for (2)?

$\endgroup$
1