7
$\begingroup$

I have a question on the slope coefficient of OLS compared to that for Quantile Regression, when facing homoscedastic error terms. The population model may look like:

$y_i = \beta_0 + \beta_{1}x_i + u_i$

with $u_i$ being iid error terms. Will the estimated slope coefficient $\hat{\beta}_{1}$ converge to the same value $\beta_{1}$ for OLS and for QR for different quantiles? While the sample estimates $\hat{\beta}_{1}$ may well be different from one another.

Considering the convergence of QR estimators, I know that in the presence of homoscedasticity all the slope parameters for different quantile regressions will converge to the same value (as shown by Koenker 2005: 12). But I am just not sure how the convergence of the OLS coeffiecient $\beta_{1}$ will compare to that of the median QR (the LAD) coefficient $\beta_{1}(0.5)$ for example. Is there a proof that both will converge to the same value? My intuition tells me this should be the case.

The answer is probably in the loss functions for OLS and QR. OLS minimizes squared residuals, while QR (for the median) minimizes absolute deviations. Therefore, as errors are squared, OLS puts more weight on outliers as opposed to QR. But in the case of homoscedasticity, shouldn't outliers cancel each other out because positive errors are as likely as negative ones, rendering OLS and median QR slope coeffiecient equivalent (at least in terms of convergence)?

Update
In order to test the prediction that for homoscedasticity the slope coefficients for different quantiles are equivalent, I ran a test in stata. This is done only to confirm the result of Koenker (2005) mentioned earlier. The original question is regarding the convergence of OLS as compared to QR. I created n=2000 observations with Stata via:

set obs 2000  
set seed 98034  
generate u = rnormal(0,8)  
generate x = runiform(0,50)
generate y = 1 + x + u

For this sample I performed a QR regression for the quantiles (0.10, 0.50, 0.90) and then tested the joint hypothesis that the slope coefficient for the three quantiles is identical, i.e.:

$H_0: \beta_1(0.1)=\beta_1(0.5)=\beta_1(0.9)$

This is the corresponding stata code:

sqreg y x, quantile(.1, .5, .9) reps(400)
test [q10=q50=q90]: x

The evidence was overwhelmingly, the H0 could very strongly not be rejected. Output for the Wald test:

F(  2,  1998) =    0.79
Prob > F =    0.4524

This reaffirmed my thoughts, but it does not provide any theoretical guidance on whether this should always be expected.

$\endgroup$
  • $\begingroup$ I am confused by your problem formulation. The point estimates are different, period. Hwowever, the estimators are consistent, so they converge in ever larger samples. Now what meaning does your hypothesis test have? You are testing whether estimates (some functions of the sample data) are somehow indistinguishable. But typically we are testing hypothesis about the population parameters. We already know everything about the sample as we observe each data point in it. I do not get what you are trying to achieve. $\endgroup$ – Richard Hardy Dec 31 '16 at 13:24
  • 1
    $\begingroup$ Thanks for your remarks, my question was really lacking some clarity. I now added the emphasis on convergence to the question. $\endgroup$ – Tartan Leaves Dec 31 '16 at 13:54
  • $\begingroup$ But in how far is the meaning of the hypothesis test unclear? If the slope coefficients for QR for different quantiles should converge to the same value, shouldn't this lead to insignificant deviations from the slope parameter among each other? Which is confirmed by the Wald test? Note however, that this is actually only a sidetrack, as my original question would be regarding convergence of QR compared with convergence of OLS. $\endgroup$ – Tartan Leaves Dec 31 '16 at 13:58
  • $\begingroup$ I am trying to tell that your null hypothesis does not make sense, at least for me. Could you write it out explicitly? Regarding convergence to the same value, this is already in my answer. Note that if two different estimators are consistent, they converge to the same value. If they converged to different values, at least one of them would be inconsistent. $\endgroup$ – Richard Hardy Dec 31 '16 at 14:13
  • $\begingroup$ I wrote down the H0 explicitly now. How sure are you, that LAD and OLS will converge to the same value? You write in your answer that "you guess" they will. $\endgroup$ – Tartan Leaves Dec 31 '16 at 14:21
5
$\begingroup$

Will the estimated slope coefficient $\beta_1$ always be the same for OLS and for QR for different quantiles?

No, of course not, because the empirical loss function being minimized differs in these different cases (OLS vs. QR for different quantiles).

I am well aware that in the presence of homoscedasticity all the slope parameters for different quantile regressions will be the same and that the QR models will differ only in the intercept.

No, not in finite samples. Here is an example taken from the help files of the "quantreg" package in R:

library(quantreg)
data(stackloss)
rq(stack.loss ~ stack.x,tau=0.50) #median (l1) regression fit for the stackloss data.
rq(stack.loss ~ stack.x,tau=0.25) #the 1st quartile

However, asymptotically they will all converge to the same true value.

But in the case of homoscedasticity, shouldn't outliers cancel each other out because positive errors are as likely as negative ones, rendering OLS and median QR slope coeffiecient equivalent?

No. First, perfect symmetry of errors is not guaranteed in any finite sample. Second, minimizing the sum of squares vs. absolute values will in general lead to different values even for symmetric errors.

$\endgroup$
  • $\begingroup$ I get from your comment the importance of distinguishing between convergence and finite sample properties. However, regarding the 2nd part of your answer, there are 2 things that are unclear to me. First, I am quite sure that slope parameters for different quantile regressions under homoscedasticity should indeed be equal. I take this certainty from Koenker (2005: 12), who notes on exactly the same model as I presented: "quantile functions are simply a vertical displacement of one another and $\hat{\beta}(\tau)$ estimates the population parameters $(\beta_0 + F^{-1}(\tau) , \beta_1)$." $\endgroup$ – Tartan Leaves Dec 31 '16 at 10:59
  • $\begingroup$ Second, regarding the equivalence of QR and OLS coefficients. Do you say that OLS and LAD will asymptotically converge to the same true value, given we have homoscedasticity? So that in finite samples the 2 may not be equivalent, but for sample sizes converging to infinity the 2 are indeed equivalent, again under the presumption of homoscedasticity? $\endgroup$ – Tartan Leaves Dec 31 '16 at 10:59
  • $\begingroup$ @TartanLeaves, Regarding comment #1: try estimating two quantile regressions on the same data set but for different quantiles, and you will see for yourself that the resulting estimates are different. This is easy to do. Regarding comment #2: Yes. In other words, both are consistent, but they will differ in finite samples. $\endgroup$ – Richard Hardy Dec 31 '16 at 11:34
  • $\begingroup$ @TartanLeaves, I have edited my answer to include an example. $\endgroup$ – Richard Hardy Dec 31 '16 at 11:44
  • $\begingroup$ I edited my original question to include a stata result. Actually I ran that experiment yesterday already. $\endgroup$ – Tartan Leaves Dec 31 '16 at 13:17
1
$\begingroup$

Generally the answer is yes, at least for Theil's regression, which is a special case of QR. The slope estimator for Theil's regression is an unbiased estimator of the population slope. If all the requirements for OLS are met, then it has 85% relative efficiency. There are certain circumstances where it becomes more efficient than least squares on a relative basis.

In addition, if you are not sitting around with an infinite amount of data, but instead have a small sample, there are many places where it would be preferable. Skew and truncation by not permitting negative values can have a strong impact on OLS and little to none on Theil's method.

$\endgroup$
  • $\begingroup$ How is the relative efficiency defined? As the ratio of asymptotic variances of the estimators? Doesn't it depend on the error distribution (nothing in the "requirements for OLS" specifies an error distribution)? (E.g. the relative efficiency of the QR to OLS does depend on the error distribution.) Also, is Theil's regression really a special case of QR? (Would you have a reference?) $\endgroup$ – Richard Hardy Dec 30 '16 at 20:00
  • $\begingroup$ I just moved and buried somewhere I have original articles both by Theil and by Pranab Sen. I also have non-parametric and distribution free textbooks buried in boxes. Theil wrote four articles that were combined into two super-articles that were part of conference proceedings, in, if memory serves me, the Royal Academy of Sciences in Denmark or Holland. He did not actually present either, someone presented for him as he had to be absent. Pranab Sen is writing generally about median based estimators. I believe Theil is in 1950 and Sen is in 1968. His was either in JASA or Econometrica. $\endgroup$ – Dave Harris Dec 30 '16 at 22:58
  • $\begingroup$ As to be a limiting case of quantile regression, it comes from a red book I have on distribution free methods. It's got white graphs on the front cover. It isn't Kendall's work. It might be by Sprent. In the case of the 85%, it assumes perfect normality. That is from Sen. DIfferent distributions end up with different relative efficiencies. I do believe it is the ratio of the asymptotic variances. $\endgroup$ – Dave Harris Dec 30 '16 at 23:02
  • $\begingroup$ Sorry, I have a weird memory for things. I can even tell you one of the other non-parametric methods covered by Sen, but not the other. I cannot tell you which box in my basement that is yet to be opened that it is in. I remember album covers, but not band or song names. Or I remember lyrics but cannot remember who sung the song. When my boxes are unpacked I will try and remember to come back to this post and update it. I can remember the JSTOR image for Sen's article on the front page, but not for the two by Theil, which may not have come through JSTOR. $\endgroup$ – Dave Harris Dec 30 '16 at 23:07
  • 1
    $\begingroup$ Thank you for your reply, @user25459! Like @Richard Hardy noted before, I am also not sure in how far Theil's regression - I guess you are referring to the 'median of pairwise slopes' method - would be considered as a special case of QR as you say. I hadn't heard of it before and in Koenker's (2005) monograph "Quantile Regression" it is only mentioned in one sentence. $\endgroup$ – Tartan Leaves Dec 31 '16 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.