3
$\begingroup$

Consider a linear model, $$ y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \epsilon_i. $$ From the Gauss-Markov theorem, I know that, under nice conditions, the $\hat{\beta}_{OLS}=(X^TX)^{-1}X^Ty$ estimate of the linear regression parameter vector $\beta$ is unbiased and has the lowest variance of all linear estimators that are unbiased. These nice conditions do not restrict the distribution of the error term, so while we often assume it to be normal, it could have much heavier tails ($t$-distributed, for example).

When I have done simulations with error distributions with heavier tails $(\epsilon_i \overset{iid}{\sim}t_{2.1}$, so variance is finite$)$, I have found that the parameter estimates for the OLS estimate have higher variance than the parameters when I do quantile regression at the median. By the Gauss-Markov theorem, $\hat{\beta}_{OLS}$ must have lower variance than any other linear, unbiased estimator, so if the quantile regression estimates have lower variance and are (I assume) linear, then the quantile regression estimates must be biased.

However, my response variable has a symmetric distribution, so mean and median are equal. What gives? This feels wrong.

$\endgroup$
1

1 Answer 1

3
$\begingroup$

so if the quantile regression estimates have lower variance and are (I assume) linear, then the quantile regression estimates must be biased.

Indeed. However, your assumption is incorrect. Quantile regression estimators are nonlinear, so they do not belong to the category of estimators (linear unbiased ones) among which OLS is "best", i.e. has then minimum variance.

$\endgroup$
8
  • $\begingroup$ Well that's extremely shocking news that I would like to revisit in a moment. First, though, should the quantile regression parameters be biased for the $\beta_0$, $\beta_1$, and $\beta_2$ in my regression equation? If the quantile regression estimator is not linear, then Gauss-Markov does not preclude unbiasedness. $\endgroup$
    – Dave
    May 21, 2020 at 19:02
  • $\begingroup$ @Dave, for these kind of queries I would take a look at the seminal paper by Koenker & Bassett "Regression quantiles" (1978) or Koenker "Quantile Regression" (2005). The latter monograph has lots of information about quantile regression, I am quite certain the answer can be found there. $\endgroup$ May 21, 2020 at 19:22
  • $\begingroup$ @Dave, hmm... A casual search within both sources did not turn out anything relevant when it comes to bias... Maybe a more careful look is needed. $\endgroup$ May 21, 2020 at 19:32
  • $\begingroup$ I did read the answer you linked, and while I do not yet fully understand it, it seems like we get an unbiased estimator when the conditions are decent. How, though, is the quantile regression estimate nonlinear? That's totally shocking. $\endgroup$
    – Dave
    May 22, 2020 at 20:27
  • $\begingroup$ How come? Have you ever seen an explicit expression of the quantile regression estimator? If it were linear, you would have, but since it is nonlinear, I guess you have not. $\endgroup$ May 22, 2020 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.