1
$\begingroup$

If you assume a Gaussian distribution for the "errors" of your regression model, you can express the maximum likelihood, so the coefficients giving the highest likelihood for the observed data, with the sum of squared "errors" of your regression model.

I am trying to calculate the AIC for my quantile regression models, and in order to do that I need to get the likelihood assuming some distribution. The distribution "associated" with the quantile loss function is the asymmetric Laplace distribution.

Of course it is possible to code the pdf and use optimization algorithms to get the maximum likelihood, but I would like to know if something similar to the Gaussian case is available. So a formula linking the maximum likelihood of the asymmetric Laplace and the "errors", or residuals rather than errors since I will be working with predicted and observed values.

I would like you to point out any nonsense terminology I might be using.

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Quantile regression and maximum likelihood estimation (MLE) are not entirely compatible from a theoretical point of view. Or, in other words when performing quantile regression you are most likely not doing MLE, but rather empirical risk minimization (ERM). This is a nuance, but MLE requires the assumption of a conditional distribution, and quantile regression does not do that. In the same sense, when performing OLS you are not necessarily doing MLE: it can also be framed as a ERM, since minimizing the MSE leads to an estimate of the expectation, but both are equivalent mathematically. $\endgroup$
    – Firebug
    Commented Jun 27, 2023 at 7:59
  • 1
    $\begingroup$ As an example, if you were given a set of data, assumed the conditional distribution was described by the asymmetric Laplace distribution and performed maximum likelihood estimation, you would also estimate the asymmetry parameter (and not vary it to estimate separate conditional quantiles) $\endgroup$
    – Firebug
    Commented Jun 27, 2023 at 8:03
  • $\begingroup$ Very interesting, thanks for pointing that out, assuming a likelihood is more restrictive since QR could indeed be framed as a pure loss minimization problem without assuming any conditional distribution. I do not know how restrictive that is practically speaking, although the example of the separate conditional quantiles being constrained by the same asymmetry parameter is telling. Do you have any posts or papers I might read on those considerations, it would be interesting. $\endgroup$
    – FaresDjerourou
    Commented Jun 27, 2023 at 8:10
  • $\begingroup$ Also that makes my issue of using the AIC or any likelihood based method to compare quantile regression models even more...delicate $\endgroup$
    – FaresDjerourou
    Commented Jun 27, 2023 at 8:22
  • 1
    $\begingroup$ This post stats.stackexchange.com/q/606949/60613 has some discussion on this $\endgroup$
    – Firebug
    Commented Jun 27, 2023 at 8:58

1 Answer 1

Reset to default
1
$\begingroup$

Maximum likelihood estimation (MLE) in regression is most often performed in a parametric fashion: you make an assumption about the conditional distribution of the dependent variable and optimize the parameters of this distribution to match the data. What you get in return is a full estimate of the conditional distribution.

Quantile regression is a non-parametric regression technique. Instead of making an assumption regarding the conditional distribution, it estimates the quantiles directly. Theoretically, it is possible to estimate as many conditional quantiles as desired (and use those to approximate a conditional distribution). Since no likelihood enters the model, there is not one afterwards to compute the AIC.

With enough resolution in the quantiles, it should be possible to obtain a non-parametric likelihood however.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.