The question comes from the paper ``Regression Quantiles'' by Roger Koenker and Gilbert Bassett(Econometrica, 1978).

$0< \theta <1$. Define $\psi(b;\theta,y,X)=\sum^{T}_{t=1}[\theta-1/2+1/2 \; \text{sgn}(y_{t}-x_{t}b)][y_{t}-x_{t}b]$, where sgn(u) takes values $1,0,-1$ as $u \gtreqless 0$.

Here, $y_{t}$ is a scalar and $x_{t}$ is a $K \times 1$ vector.

Then \textbf{my question} is: what is the directional derivative of $\psi(b)$ in the direction $w$?

The paper states that the answer is \begin{equation} \psi^{\prime}(b;w)=\sum^{T}_{t=1}[1/2-1/2 \; \text{sgn}^{\ast}(y_{t}-x_{t}b;-x_{t}w)-\theta]x_{t}w, \end{equation} where $\text{sgn}^{\ast}(u;z)= \begin{cases} \text{sgn} \; u \; \text{if} \; u \neq 0,\\ \text{sgn} \; z \; \text{if} \; u = 0 \end{cases}$

But I do not know how to derive this result.


First things first, please add transpose to some symbols whenever it is necessary.

Thanks for raising this question, myself tried to verify this when I was reading Koenker's monograph $Quantile\; Regression$. In general, the expression he displayed was correct, but I think he didn't write everything clearly (at least lacking of necessary explanation for some part) so I would like to present my own derivation here and call for more discussion. Given a multivariate-variable and real-valued function $f(x_1, \ldots, x_n)$, the directional derivative of $f$ along a direction $\mathbf{w}$ at $\textbf{x}$ is defined to be: $$\nabla_\mathbf{w}f(\mathbf{x}) = \lim_{h\to 0}\frac{f(\mathbf{x} + h\mathbf{w}) - f(\mathbf{x})}{h}$$ We shall verify the claimed result following the above definition. To alleviate the notation, denote $y_t - x'b$ by $u_t$, $\psi(b + hw; \theta, y, X)$ by $f(h)$, we want to show the limit of $(f(h) - f(0))/h$ exists and has the claimed form.

Straightforward algebra shows: $$f(h) - f(0) = \sum_{t = 1}^T\left[-(u_t - hx_t'w)\mathrm{I}\{u_t - hx_t'w < 0\} - hx_t'w\theta + u_t\mathrm{I}\{u_t < 0\}\right]$$ where $\mathrm{I}(\cdot)$ is the indicator function.

We may now evaluate the limit term-wisely, by discussing the positivity of $u_t$:

If $u_t > 0$, then the summand is $-hx_t'w\theta$ (the indicator function always takes zero when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T-\theta x_t'w$$

If $u_t < 0$, then the summand is $(1 - \theta)hx_t'w$ (the indicator function always takes one when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T(1 - \theta) x_t'w$$

For these two cases, the declared expression form were all successfully confirmed. However, the annoying and unclear case is that $u_t = 0$, for which, the summand is: $$(\mathrm{I}\{-hx_t'w < 0\} - \theta)hx_t'w$$ Let $g(h) = (\mathrm{I}\{-hx_t'w < 0\} - \theta)x_t'w$, namely, the function after the above expression divided by $h$. Clearly, $g(h)$ is not differentiable at $0$. Hence it is very irresponsible to just state the result without any further explanation. We may discuss the right/left limit of $g(h)$ as $h \to 0$, which depends on the value of $x_t'w$. $$\lim_{h \to 0^+} g(h) = (1/2 - 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w \\ \lim_{h \to 0^-} g(h) = (1/2 + 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w$$

So the result reported by the author is essentially the "right directional derivative" instead of "directional derivative".


The objective function for quantile regression is a weighted sum of the absolute deviations, where the weights depend on the quartile $\tau$: $$R(\beta_\tau)=\sum_i^N \rho_\tau\cdot \vert y_i-x_i'\beta_\tau\vert=\sum_{y_i \ge x_i'\beta_\tau} \tau \cdot (y_i-x_i'\beta_\tau ) +\sum_{y_i<x_i'\beta_\tau} (\tau-1) \cdot ( y_i-x_i'\beta_\tau ).$$

The directional derivative of this puppy in direction $w$ is

$$\nabla R(\beta_\tau) = \frac{\partial \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (y_i-x_i'(\beta_\tau +tw) )}{\partial t} \big|_{t=0}+\frac{\partial \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot ( y_i-x_i'(\beta_\tau +tw))}{\partial t}\big|_{t=0}. $$ which is equal to

$$ \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (-x_i'w) + \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot (-x_i'w), $$

which I believe is equivalent to what you have above.

  • $\begingroup$ Hi Dimitriy. Nice to meet you as an economist here. Thank you for your reply. Assuming that the paper "Regression Quantiles" is familiar with you, would you like to answer my other concern at stats.stackexchange.com/questions/134338/…? Thank you. $\endgroup$
    – Jie Wei
    Jan 21 '15 at 10:38

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .