23
$\begingroup$

I am trying to learn more about Quantile Regression.

As I understand, Quantile Regression is used to estimate the conditional quantile of a response variable (given predictor variables).

Mathematically, let $Y$ be the response variable and $X$ be a vector of predictor variables. The $\tau$-th quantile regression model can be written as:

$$ Q_{Y|X}(\tau) = X\beta(\tau) $$

where $Q_{Y|X}(\tau)$ is the conditional quantile function of $Y$ given $X$, and $\beta(\tau)$ is a vector of unknown parameters that depend on the quantile $\tau$. The goal of quantile regression is to estimate the parameters $\beta(\tau)$ for a given value of $\tau$.

I am trying to understand: For what kinds of problems is Quantile Regression better suited?

When I asked my teacher in school, my teacher indicated that Quantile Regression is intended for applications where you might be specifically interested in modelling the effect of the predictors on some quantile of the response (e.g. median response) instead of the mean response.

But I am trying to understand - in what types of situations would you specifically be interested in modelling a Quantile of the Response Variable instead of the Mean Response? Are there some industries/domains where this requirement naturally arises?

The closest thing which comes to mind is situations where the conditional distribution of the response given the predictor variables might be heavily skewed , partly violating the assumptions of standard regression. In such a case, I think it might somehow be more useful to model some quantile of the response (via Quantile Regression) instead of the mean response. Is this reasoning correct?

$\endgroup$
6
  • 5
    $\begingroup$ “Prof, when do we want to model quantiles?” “When we’re interested in the quantiles.” Just phenomenal teaching. $//$ You have an academic affiliation, so you should be able to access journal articles. Koenker is the original developer of quantile regression. You might be interested in the motivation given in his work. $\endgroup$
    – Dave
    Jul 29, 2023 at 2:47
  • 1
    $\begingroup$ QR generates much more useful predictions wrt extreme-valued information insofar as one can specify quantiles widely diverging from the mean or median. $\endgroup$
    – user78229
    Jul 29, 2023 at 12:21
  • 3
    $\begingroup$ (A) When the median gives a better summary than the mean. (B) When you are interested in the full conditional distribution of the response. (C) When you need a simple way to calculate 80% prediction intervals (slightly optimistic). $\endgroup$
    – Michael M
    Jul 29, 2023 at 19:52
  • 1
    $\begingroup$ @MichaelM the problem with linear QR and your point (B) is that it does not guarantee proper conditional distributions (e.g. crossing quantiles) $\endgroup$
    – Firebug
    Jul 30, 2023 at 8:16
  • 1
    $\begingroup$ @Firebug, good point. Perhaps composite quantile regression (a collection of QRs for different quantiles with a restriction that the slopes are common across quantiles) could fix that problem. $\endgroup$ Jul 30, 2023 at 15:43

5 Answers 5

21
$\begingroup$

One example from tech is p90 of website load speed or service response time. This analysis also comes up in logistics. Here you care about just how bad the worst experience is. Typically this is something you monitor as you make changes, with alerts that fire when that quantile rises above some threshold. Even if, on average, the website loads quickly or purchases arrive the next day, you want to know about the worst scenarios.

In education research, you often care about how an intervention like an AI tutor affects different types of students. Does it move up the left tail (low test scores) or the right tail (helping highest-scoring students)?

The common theme in many such applications is that you are not interested in the conditional mean or how it changes, but in other parts of the distribution.

$\endgroup$
19
$\begingroup$

Quantile regression is used for modeling growth curves of children. A pediatrician is not surprised if a child is below the median height as a function of age but starts to worry if the child's height is below the 10th percentile of height given age. Quantile regression allows estimation of quantiles without assuming a distribution.

But there are two catches. Quantile regression needs the response variable to be very continuous (i.e., few ties), and the efficiency of the estimates is not great. In the simplest case with no predictors and one predicts the median, quantile regression gives exactly the sample median, which has efficiency $\frac{2}{\pi}$ compared to using the mean, which also estimates the median if the distribution happened to be symmetric. So one could say that quantile regression is for larger sample sizes.

Semiparametric ordinal regression models can also estimate quantiles without assuming a distribution, they allow for arbitrarily heavy ties in the data, and they can also estimate the mean and the entire cumulative distribution of Y. See here for resources. Semiparametric models are very efficient as compared to parametric models.

$\endgroup$
3
  • $\begingroup$ In the link, what do you mean in that last comment about JMP, that JMP can handle a case that rms::orm cannot? $\endgroup$
    – Dave
    Jul 29, 2023 at 18:19
  • 1
    $\begingroup$ Please re-read the text. It says that only R rms::orm and JMP have programmed a sparse matrix solution that efficiently computes the needed quantities when there are thousands of intercepts. The two software implementations use similar sparse matrix methods. $\endgroup$ Jul 29, 2023 at 18:42
  • $\begingroup$ This example acutally goes further: If a child is at the 67. percentile of body height at their birth, they will stay on that percentile at least for their first two years. This even be used to predict their clothes size. A calculator is here (German only, unfortunately): btelligent.com/baby-kleidergroessenrechner $\endgroup$ Aug 1, 2023 at 9:23
8
$\begingroup$

Mentioned in dimitriy's answer, but deserving of a little more detail: if you want to find a target inventory or capacity, you will typically not only want to satisfy average demand, but achieve a higher service level. (Note that there are different definitions of "service level" in inventory theory, not all of which make sense in any given situation.)

For instance, you might run a quantile regression in order to forecast what the conditional 95th percentile of total demand in a supermarket is, where you condition on predictors like day of week, day of year, price or promotional activity. This quantile prediction then becomes your target inventory. (Whether the 95th percentile is actually the one to maximize your profit, or maybe you had better aim for a 90th or 98th percentile, is a difficult question that depends on your logistics and on the behavior of your customers. The question is easy to answer in a newsvendor situation, which is not always what you are facing.)

This is why the recent M5 forecasting competition, which explicitly focused on retail forecasting and used Walmart data, required submitting quantile forecasts in its "uncertainty" track.

$\endgroup$
3
+50
$\begingroup$

Another answer, straight from a real-life project: Imagine you're an e-commerce seller on a large platform (think Amazon, Rakuten, or whatever is popular in your area). Now you want a data-driven solution for telling your customer for each of your products how long it will take to deliver. You could just do a regression on past delivery times for this kind of item, right?

But there's a catch: These platforms usually require their sellers to sign service level agreements, and one of the things they usually have to commit to is that at least 90% of their shipments (or some other share, this is highly country-specific) arrive on time, i.e. the actual delivery time is smaller or equal to what was promised. So if you want a result that complies with the contract you've signed, you want to use a 90%-quantile regression.

$\endgroup$
2
  • $\begingroup$ What does the p90 regression give you compared with calculating the on-time-or-sooner delivery rates for each seller? $\endgroup$
    – dimitriy
    Aug 16, 2023 at 7:26
  • 1
    $\begingroup$ @dimitriy The on-time-or-sooner-rate can only be calculated in retrospective. If you want a prediction, you need regression. If you want a prediction that leads to at least 90% of shipments arriving at the predicted time or earlier, you need quantile regression. $\endgroup$ Aug 28, 2023 at 12:00
2
$\begingroup$

Some practical examples where I have used quantile reg:

Neonatologists and obstetricians are interested in predicting the birth weight of babies. But they are especially interested in very small and very large babies, as this is where problems are most common (e.g. the baby needing emergency care, or problems in labor). I ran quantile reg. on low and high quantiles and found that the predictors were quite different from what I got for OLS regression.

Epidemiologists are interested in the spread of disease. I worked for a long time at a place that did research into the spread of HIV/AIDS. One way that these spread is through sex, so, we were interested in people who had sex with a lot of partners. Trying to predict this turned out to be quite different from predicting the mean number of partners. (This is also a case where the errors are not remotely normal).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.