It is easy to show using matrix algebra when least squares will produce bias.

\begin{equation} \begin{split} \text{E}[B]& = \text{E}[(X'X)^{-1}]\times\text{E}[X'Y] \\ & = \text{E}[(X'X)^{-1}]\times\text{E}[X'(XB +\epsilon)] \\ & = \text{E}[(X'X)^{-1}]\times\text{E}[X'XB] + \text{E}[(X'X)^{-1}]\times\text{E}[X'\epsilon] \\ & = B + \text{E}[(X'X)^{-1}]\times\text{E}[X'\epsilon] \\ \end{split} \end{equation}

Bias is introduced in the last term when there is correlation between X and e.

My question is, how do we know when $\text{E}[B]$ for the LAD estimator will be biased? Will bias arise when there is correlation between $X$ and $\epsilon$, as in least squares? Or is quantile regression robust to correlation between $X$ and $\epsilon$? I'm guessing it can't be demonstrated using matrix algebra because the LAD estimator is the following:

\begin{align} B(\tau) = \operatorname{argmin} \text{E}[\rho(Y_i - X'B)] \end{align}

and the LAD estimator is not calculated using linear algebra (?). If that's right, then how can we demonstrate when quantile regression will produce bias? Use a monte carlo simulation?

  • $\begingroup$ Please clarify. If we condition on $X$, $X$ is non-stochastic and doesn't correlate with a random variable in the usual sense. $\endgroup$ – Frank Harrell Jul 30 '13 at 20:06
  • 1
    $\begingroup$ What are your assumptions? It is almost always the case that $\mathbb{E}[e]=0$, whence your bias term reduces to zero in the usual application where $X$ is fixed. But if you're viewing $X$ as random, your expression for the expectation doesn't simplify in the way you write. $\endgroup$ – whuber Jul 30 '13 at 20:07
  • $\begingroup$ You should have a look at . CHernozhukov and Hansen "Quantile Models with Endogeneity", 2012, forthcoming Annual Review of Economics : mit.edu/~vchern/papers/IVQRReview5.pdf $\endgroup$ – PAC Jul 30 '13 at 21:55

If you have a model $$ Y_i = X_i B(\tau) + \epsilon_i(\tau) $$ then a sufficient condition for $\tau$-quantile regression to give an unbiased estimate of $B(\tau)$ is that the $\tau$-th quantile of $\epsilon(\tau)$ conditional on $X$ is zero. This follows from the fact that (i) the sample quantile regression objective function, $\mathbb{E}_n[ \rho_\tau(Y-X'\beta)]$ converges uniformly to $\mathrm{E}[\rho_\tau(Y-X'\beta)]$ and (ii) $\mathrm{E}[\rho_\tau(Y-X'\beta)]$ is "uniquely" (actually something slightly stronger is needed) minimized at $B(\tau)$. (i) will be true under standard regularity conditions. (ii) is true because $\mathrm{E}[\rho_\tau(Y-X'\beta)]$ is convex as a function of $\beta$ and its first order condition can be written $$ \tau \mathrm{E}[ P(Y - X\beta>0|X) X] = (1-\tau) \mathrm{E}[ P(Y - X\beta<0|X) X] $$ which is satisfied at $\beta = B(\tau)$ if $Q_\tau(\epsilon(\tau)|X) = 0$. You can also see from this that $Q_\tau(\epsilon(\tau)|X) = 0$ is stronger than needed, but there doesn't seem to be an easy to interpret weaker condition.

None of the above tells you what the bias in $\hat{B}(\tau)$ would be if $Q_\tau(\epsilon(\tau)|X) \neq 0$. I don't know of a general expression for the bias like you can get for OLS, but you can get some nice results in a few cases. For example, Angrist, Chernozhukov, and Fernandez-Val (2006) give an omitted variables bias formula for quantile regression. If your model satisfies the conditions above, and $X = (X_1, X_2)$, and then you estimate a quantile regression leaving out $X_2$, then the expectation of your estimated coefficient on $X_1$ is $$ \beta_1(\tau) + \mathrm{E}[w_\tau(X) X_1'X_1]^{-1} \mathrm{E}[w_\tau(X)X_1' (X_2' \beta_2(\tau))] $$ where $w_\tau(X)$ are some weights that depend on $X$, $\tau$, and the distribution of $\epsilon$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.