24
$\begingroup$

Apart from some unique circumstances where we absolutely must understand the conditional mean relationship, what are the situations where a researcher should pick OLS over Quantile Regression?

I don't want the answer to be "if there is no use in understanding the tail relationships", as we could just use median regression as the OLS substitute.

$\endgroup$
1
  • 5
    $\begingroup$ I think most researchers would entertain both OLS and quantile regression; differences between the methods would shine light on what you are trying to model. With respect to OLS, if you toss in normality assumptions you do get a lot of fairly well documented and thorough testing methodology that is available in most statistical packages. $\endgroup$ Feb 1 '13 at 21:06
22
$\begingroup$

If you are interested in the mean, use OLS, if in the median, use quantile.

One big difference is that the mean is more affected by outliers and other extreme data. Sometimes, that is what you want. One example is if your dependent variable is the social capital in a neighborhood. The presence of a single person with a lot of social capital may be very important for the whole neighborhood.

$\endgroup$
2
  • 7
    $\begingroup$ Let me challenge your first sentence. Both OLS and quantile regression (QR) are estimating $\beta$ for a data generating process $y=X\beta+\varepsilon$. If the error distribution has heavy tails, $\hat\beta^{QR}$ is more efficient than $\hat\beta^{OLS}$. Regardless of which moment of the conditional distribution $P(y|X)$ we are interested in, we should use the one of $\hat\beta^{OLS}$ and $\hat\beta^{QR}$ that is more efficient. $\endgroup$ Dec 14 '16 at 14:23
  • $\begingroup$ Following on @RichardHardy 's critique of this response the median is only one of the quantiles that are estimable. This Hyndman paper introduces an approach he calls boosting additive quantile regression which explores a full range of quantiles, Forecasting Uncertainty in Electricity Smart Meter Data by Boosting Additive Quantile Regression (ieeexplore.ieee.org/document/7423794). $\endgroup$ May 1 '18 at 13:07
19
$\begingroup$

There seems to be a confusion in the premise of the question. In the second paragraph it says, "we could just use median regression as the OLS substitute". Note that regressing the conditional median on X is (a form of) quantile regression.

If the error in the underlying data generating process is normally distributed (which can be assessed by checking if the residuals are normal), then the conditional mean equals the conditional median. Moreover, any quantile you may be interested in (e.g., the 95th percentile, or the 37th percentile), can be determined for a given point in the X dimension with standard OLS methods. The main appeal of quantile regression is that it is more robust than OLS. The downside is that if all assumptions are met, it will be less efficient (that is, you will need a larger sample size to achieve the same power / your estimates will be less precise).

$\endgroup$
14
$\begingroup$

Both OLS and quantile regression (QR) are estimation techniques for estimating the coefficient vector $\beta$ in a linear regression model $$ y = X\beta + \varepsilon $$ (for the case of QR see Koenker (1978), p. 33, second paragraph).

For certain error distributions (e.g. those with heavy tails), the QR estimator $\hat\beta_{QR}$ is more efficient than the OLS estimator $\hat\beta_{OLS}$; recall that $\hat\beta_{OLS}$ is efficient only in the class of linear unbiased estimators. This is the main motivation for Koenker (1978) that suggests using the QR in place of OLS under a variety of settings. I think that for any moment of the conditional distribution $P_Y(y|X)$ we should use the one of $\hat\beta_{OLS}$ and $\hat\beta_{QR}$ that is more efficient (please correct me if I am wrong).

Now to answer your question directly, QR is "worse" than OLS (and thus $\hat\beta_{OLS}$ should be preferred over $\hat\beta_{QR}$) when $\hat\beta_{OLS}$ is more efficient than $\hat\beta_{QR}$. One such example is when the error distribution is Normal.

References:

  • Koenker, Roger, and Gilbert Bassett Jr. "Regression quantiles." Econometrica: Journal of the Econometric Society (1978): 33-50.
$\endgroup$
5
$\begingroup$

To say what some of the excellent responses above said, but in a slightly different way, quantile regression makes fewer assumptions. On the right hand side of the model the assumptions are the same as with OLS, but on the left hand side the only assumption is continuity of the distribution of $Y$ (few ties). One could say that OLS provides an estimate of the median if the distribution of residuals is symmetric (hence median=mean), and under symmetry and not-too-heavy tails (especially under normality), OLS is superior to quantile regression for estimating the median, because of much better precision. If there is only an intercept in the model, the quantile regression estimate is exactly the sample median, which has efficiency of $\frac{2}{\pi}$ when compared to the mean, under normality. Given a good estimate of the root mean squared error (residual SD) you can use OLS parametrically to estimate any quantile. But quantile estimates from OLS are assumption-laden, which is why we often use quantile regression.

If you want to estimate the mean, you can't get that from quantile regression.

If you want to estimate the mean and quantiles with minimal assumptions (but more assumptions than quantile regression) but have more efficiency, use semiparametric ordinal regression. This also gives you exceedance probabilities. A detailed case study is in my RMS course notes where it is shown on one dataset that the average mean absolute estimation error over several parameters (quantiles and mean) is achieved by ordinal regression. But for just estimating the mean, OLS is best and for just estimating quantiles, quantile regression was best.

Another big advantage of ordinal regression is that it is, except for estimating the mean, completely $Y$-transformation invariant.

$\endgroup$
3
$\begingroup$

Peter Flom had a great and concise answer, I just want to expand it. The most important part of the question is how to define "worse".

In order to define worse, we need to have some metrics, and the function to calculate how good or bad the fittings are called loss functions.

We can have different definitions of the loss function, and there is no right or wrong on each definition, but different definition satisfy different needs. Two well known loss functions are squared loss and absolute value loss.

$$L_{sq}(y,\hat y)=\sum_i (y_i-\hat y_i)^2$$ $$L_{abs}(y,\hat y)=\sum_i |y_i-\hat y_i|$$

If we use squared loss as a measure of success, quantile regression will be worse than OLS. On the other hand, if we use absolute value loss, quantile regression will be better.

Which is what Peter Folm's answer:

If you are interested in the mean, use OLS, if in the median, use quantile.

$\endgroup$
1
  • $\begingroup$ I think your example may be misleading since it addresses in-sample fit (which is of little interest since we already know our sample perfectly) rather than expected loss for new observations (when the goal is prediction) or loss of estimating the parameter vector (when the goal is explanation). See may comment under Peter Flom's answer and my answer for more details. $\endgroup$ Jan 27 '17 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy