0
$\begingroup$

In short, I want to understand why using Laplacian distribution to model errors performs better than using Gaussian distribution to model errors. In detail, I am using Generalized normal distribution as error model for the following nonlinear regression problem.

$y_i=\hat{y_i}+\epsilon_i$

$\epsilon\sim Gen(0,\sigma_i,\beta)$

Which results in the following negative-log likelihood function

$L=0.5*\sum_{i=1}^{N}\ln(\sigma_i^2)+c(\beta)*\sum_{i=1}^{N}|\frac{y_i-\hat{y_i}}{\sigma_i}|^{\frac{2}{(1+\beta)}}$

where

$c(\beta)=\lbrace\frac{\Gamma[1.5*(1+\beta)]}{\Gamma[0.5*(1+\beta)]}\rbrace^{\frac{1}{1+\beta}}$

$Gen(\mu,\sigma,\beta)$ implies generlized normal distribution with mean$\mu$, standard deviation $\sigma$ and shape parameter $\beta$; $y_i$ is the $i^{th}$ observation; $\hat{y_i}$ is the model prediction of the $i^{th}$ observation; $N$ is the total number of observations; $\sigma_i^2$ is the variance of the $i^{th}$ prediction.

In this model, $\beta$ determines the kurtosis of the distribution. For example, $\beta=1$ implies that $L$ is the negative-log likelihood of the laplacian (double exponential) error distribution and $\beta=0$ implies that the L is the negative-log likelihood of the Normal error distribution. I get significantly better performance when I use Laplacian distribution for error modeling. What could be the possible reasons for this? I know that Laplacian distribution error model is robust to outliers, but I don't have any outliers in my data. It will be very helpful if someone can point me to some text discussing the related issues.

$\endgroup$
  • $\begingroup$ See for instance stats.stackexchange.com/questions/214617/… $\endgroup$ – kjetil b halvorsen Mar 29 '18 at 14:17
  • 1
    $\begingroup$ As you note, the Laplace distribution one particular instance of a Generalized Normal. You therefore seem to be asking why one instance of your chosen model works better than other instances--but isn't it always going to be the case that one such instance works better (has larger likelihood) than most others? What kind of "reason" are you seeking? $\endgroup$ – whuber Mar 29 '18 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.