I am familiarizing myself with quantile regression. I understand it is first and foremost an estimation method such as e.g. OLS. But I wonder about the probability distribution models for which quantile regression makes sense.

To use an analogy, the OLS estimator $$ \hat\beta^{OLS}:=(X^tX)^{-1}X^ty $$ is a minimum variance linear unbiased estimator when the true probability distribution model is $$ \begin{align} y &= X\beta+\varepsilon, \\ \varepsilon &\sim d(0,\sigma^2) \end{align} $$ (or in other words, $y|X\sim d(X\beta,\sigma^2)$), where $d$ is some (unspecified) probability distribution.
Moreover, $\hat\beta^{OLS}$ is a maximum likelihood estimator when the probability model is $$ \begin{align} y &= X\beta+\varepsilon, \\ \varepsilon &\sim N(0,\sigma^2) \end{align} $$ (or in other words, $y|X\sim N(X\beta,\sigma^2)$). So in some sense (loosely speaking), the OLS estimator "naturally implies" the above probability distribution models.

Question: What could we say about the quantile estimator $$ \hat\beta^{QR}_{\tau}:= \arg\min_{\beta}\sum_{i=1}^{n}\rho_{\tau}(y_i-X_i\beta) $$ where $\rho_{\tau}$ is the quantile loss function (and the subscript $^{QR}$ stands for "quantile regression")? Does it "naturally imply" a probability distribution model for $y$?

If we assumed particular true conditional quantiles $$ \beta^{QR}_{\tau}:= \arg\min_{\beta}\mathbb{E}\left(\rho_{\tau}(y-X\beta)\right) $$ (where $\beta_{\tau}$ may differ across different values of $\tau$) for a continuum of quantiles between 0 and 1, we would get an implicit conditional probability model for $y|X$.
But could the model be expresed explicitly in a nice way? If so, I would welcome a simple example.

P.S. In principle, the question covers more than just linear quantile regressions, but for practical purposes an answer addressing just the linear quantile regression would also suffice.

  • $\begingroup$ In books I've read, it's been treated as a non parametric regression but apparently the quantile estimator is the maximum likelihood estimator of an asymmetric double exponential distribution. See here web1.sph.emory.edu/users/hwu30/teaching/statcomp/Notes/… $\endgroup$ – machazthegamer Dec 28 '18 at 19:50
  • $\begingroup$ @machazthegamer, interesting. For a single quantile level $\tau$, your comment (slightly expanded) could serve as an answer. For multiple quantiles $\tau_i$ (a finite number of them) with different corresponding true $\beta^{QR}_{\tau_i}$, the model is probably underdetermined. $\endgroup$ – Richard Hardy Dec 28 '18 at 21:55
  • $\begingroup$ @machazthegamer, I would like to accept your answer if you post your extended comment as one. $\endgroup$ – Richard Hardy Jan 15 '19 at 16:49
  • $\begingroup$ Could someone expound on what was said here? $\endgroup$ – Rylan Schaeffer Jun 23 at 3:18
  • $\begingroup$ @RylanSchaeffer, where exactly? $\endgroup$ – Richard Hardy Jun 23 at 5:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.