The question comes from the paper ``Regression Quantiles'' by Roger Koenker and Gilbert Bassett(Econometrica, 1978).

$0< \theta <1$. Define $\psi(b;\theta,y,X)=\sum^{T}_{t=1}[\theta-1/2+1/2 \; \text{sgn}(y_{t}-x_{t}b)][y_{t}-x_{t}b]$, where sgn(u) takes values $1,0,-1$ as $u \gtreqless 0$.

Here, $y_{t}$ is a scalar and $x_{t}$ is a $K \times 1$ vector.

Then \textbf{my question} is: what is the directional derivative of $\psi(b)$ in the direction $w$?

The paper states that the answer is \begin{equation} \psi^{\prime}(b;w)=\sum^{T}_{t=1}[1/2-1/2 \; \text{sgn}^{\ast}(y_{t}-x_{t}b;-x_{t}w)-\theta]x_{t}w, \end{equation} where $\text{sgn}^{\ast}(u;z)= \begin{cases} \text{sgn} \; u \; \text{if} \; u \neq 0,\\ \text{sgn} \; z \; \text{if} \; u = 0 \end{cases}$

But I do not know how to derive this result.


2 Answers 2


First things first, please add transpose to some symbols whenever it is necessary.

Thanks for raising this question, myself tried to verify this when I was reading Koenker's monograph Quantile Regression. In general, the expression he displayed was correct, but I think he didn't write everything clearly (at least lacking necessary explanations for some parts) so I would like to present my own derivation here and call for more discussion.

Given a multivariate-variable and real-valued function $f(x_1, \ldots, x_n)$, the directional derivative of $f$ along a direction $\mathbf{w}$ at $\textbf{x}$ is defined to be: $$\nabla_\mathbf{w}f(\mathbf{x}) = \lim_{h\to 0}\frac{f(\mathbf{x} + h\mathbf{w}) - f(\mathbf{x})}{h}$$ We shall verify the claimed result following the above definition. To alleviate the notation, denote $y_t - x'b$ by $u_t$, $\psi(b + hw; \theta, y, X)$ by $f(h)$, we want to show the limit of $(f(h) - f(0))/h$ exists and has the claimed form.

Straightforward algebra shows: $$f(h) - f(0) = \sum_{t = 1}^T\left[-(u_t - hx_t'w)\mathrm{I}\{u_t - hx_t'w < 0\} - hx_t'w\theta + u_t\mathrm{I}\{u_t < 0\}\right]$$ where $\mathrm{I}(\cdot)$ is the indicator function.

We may now evaluate the limit term-wisely, by discussing the positiveness of $u_t$:

If $u_t > 0$, then the summand is $-hx_t'w\theta$ (the indicator function always takes zero when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T-\theta x_t'w$$

If $u_t < 0$, then the summand is $(1 - \theta)hx_t'w$ (the indicator function always takes one when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T(1 - \theta) x_t'w$$

For these two cases, the declared expression form were all successfully confirmed. However, the annoying and unclear case is that $u_t = 0$, for which, the summand is: $$(\mathrm{I}\{-hx_t'w < 0\} - \theta)hx_t'w$$ Let $g(h) = (\mathrm{I}\{-hx_t'w < 0\} - \theta)x_t'w$, namely, the function after the above expression divided by $h$. Clearly, $g(h)$ is not differentiable at $0$. Hence it is very irresponsible to just state the result without any further explanation. We may discuss the right/left limit of $g(h)$ as $h \to 0$, which depends on the value of $x_t'w$. $$\lim_{h \to 0^+} g(h) = (1/2 - 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w \\ \lim_{h \to 0^-} g(h) = (1/2 + 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w$$

So the result reported by the author is essentially the "right directional derivative" instead of "directional derivative".

  • $\begingroup$ Thank you very much, Zhanxiong, for your thorough derivation. I am have another related issue. Could you take a look at it when you have a chance? stats.stackexchange.com/questions/134338/… $\endgroup$
    – Jie Wei
    Commented Jan 21, 2015 at 10:36

The objective function for quantile regression is a weighted sum of the absolute deviations, where the weights depend on the quartile $\tau$: $$R(\beta_\tau)=\sum_i^N \rho_\tau\cdot \vert y_i-x_i'\beta_\tau\vert=\sum_{y_i \ge x_i'\beta_\tau} \tau \cdot (y_i-x_i'\beta_\tau ) +\sum_{y_i<x_i'\beta_\tau} (\tau-1) \cdot ( y_i-x_i'\beta_\tau ).$$

The directional derivative of this puppy in direction $w$ is

$$\nabla R(\beta_\tau) = \frac{\partial \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (y_i-x_i'(\beta_\tau +tw) )}{\partial t} \big|_{t=0}+\frac{\partial \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot ( y_i-x_i'(\beta_\tau +tw))}{\partial t}\big|_{t=0}. $$ which is equal to

$$ \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (-x_i'w) + \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot (-x_i'w), $$

which I believe is equivalent to what you have above.

  • $\begingroup$ Hi Dimitriy. Nice to meet you as an economist here. Thank you for your reply. Assuming that the paper "Regression Quantiles" is familiar with you, would you like to answer my other concern at stats.stackexchange.com/questions/134338/…? Thank you. $\endgroup$
    – Jie Wei
    Commented Jan 21, 2015 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.