2
$\begingroup$

Consider a linear model, $$ y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \epsilon_i. $$ From the Gauss-Markov theorem, I know that, under nice conditions, the $\hat{\beta}_{OLS}=(X^TX)^{-1}X^Ty$ estimate of the linear regression parameter vector $\beta$ is unbiased and has the lowest variance of all linear estimators that are unbiased. These nice conditions do not restrict the distribution of the error term, so while we often assume it to be normal, it could have much heavier tails ($t$-distributed, for example).

When I have done simulations with error distributions with heavier tails $(\epsilon_i \overset{iid}{\sim}t_{2.1}$, so variance is finite$)$, I have found that the parameter estimates for the OLS estimate have higher variance than the parameters when I do quantile regression at the median. By the Gauss-Markov theorem, $\hat{\beta}_{OLS}$ must have lower variance than any other linear, unbiased estimator, so if the quantile regression estimates have lower variance and are (I assume) linear, then the quantile regression estimates must be biased.

However, my response variable has a symmetric distribution, so mean and median are equal. What gives? This feels wrong.

Thanks!

$\endgroup$
2
$\begingroup$

so if the quantile regression estimates have lower variance and are (I assume) linear, then the quantile regression estimates must be biased.

Indeed. However, your assumption is incorrect. Quantile regression estimators are nonlinear, so they do not belong to the category of estimators (linear unbiased ones) among which OLS is "best", i.e. has then minimum variance.

$\endgroup$
  • $\begingroup$ Well that's extremely shocking news that I would like to revisit in a moment. First, though, should the quantile regression parameters be biased for the $\beta_0$, $\beta_1$, and $\beta_2$ in my regression equation? If the quantile regression estimator is not linear, then Gauss-Markov does not preclude unbiasedness. $\endgroup$ – Dave May 21 '20 at 19:02
  • $\begingroup$ @Dave, for these kind of queries I would take a look at the seminal paper by Koenker & Bassett "Regression quantiles" (1978) or Koenker "Quantile Regression" (2005). The latter monograph has lots of information about quantile regression, I am quite certain the answer can be found there. $\endgroup$ – Richard Hardy May 21 '20 at 19:22
  • $\begingroup$ @Dave, hmm... A casual search within both sources did not turn out anything relevant when it comes to bias... Maybe a more careful look is needed. $\endgroup$ – Richard Hardy May 21 '20 at 19:32
  • $\begingroup$ I did read the answer you linked, and while I do not yet fully understand it, it seems like we get an unbiased estimator when the conditions are decent. How, though, is the quantile regression estimate nonlinear? That's totally shocking. $\endgroup$ – Dave May 22 '20 at 20:27
  • $\begingroup$ How come? Have you ever seen an explicit expression of the quantile regression estimator? If it were linear, you would have, but since it is nonlinear, I guess you have not. $\endgroup$ – Richard Hardy May 22 '20 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.