Why is Gaussian distribution used for Maximum Likelihood estimation with Linear Regression and not some other distribution? I know that using Gaussian distribution for the target y yields Maximum Likelihood giving Mean Square Error as the loss function to be minimized. But, why use Gaussian distribution and not other distribution and try to maximize its likelihood?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ There's nothing stopping you choosing another distribution. Regression models using other conditional distributions are certainly used in practice. For example: 1. Regression with Laplace errors, for which MLE is L1 regression 2. Generalized linear models, which are ML for distributions in the exponential family. If you choose an identity link you have a model where the conditional mean is linear in the predictors 3. M-estimators for which the $\rho$ function is the negative log of an actual density 4. regression using t-errors, which crop up in a number of applications $\endgroup$– Glen_bCommented Sep 25, 2018 at 13:42
-
$\begingroup$ There's also nothing stopping you optimizing another loss function even if it isn't ML for some distribution. $\endgroup$– Glen_bCommented Sep 25, 2018 at 13:55
Add a comment
|