A question in directional derivatives of a quantile regression object function

Question

The question comes from the paper ``Regression Quantiles'' by Roger Koenker and Gilbert Bassett(Econometrica, 1978).

$0< \theta <1$. Define $\psi(b;\theta,y,X)=\sum^{T}_{t=1}[\theta-1/2+1/2 \; \text{sgn}(y_{t}-x_{t}b)][y_{t}-x_{t}b]$, where sgn(u) takes values $1,0,-1$ as $u \gtreqless 0$.

Here, $y_{t}$ is a scalar and $x_{t}$ is a $K \times 1$ vector.

Then \textbf{my question} is: what is the directional derivative of $\psi(b)$ in the direction $w$?

The paper states that the answer is \begin{equation} \psi^{\prime}(b;w)=\sum^{T}_{t=1}[1/2-1/2 \; \text{sgn}^{\ast}(y_{t}-x_{t}b;-x_{t}w)-\theta]x_{t}w, \end{equation} where $\text{sgn}^{\ast}(u;z)= \begin{cases} \text{sgn} \; u \; \text{if} \; u \neq 0,\\ \text{sgn} \; z \; \text{if} \; u = 0 \end{cases}$

But I do not know how to derive this result.

Answer 1

First things first, please add transpose to some symbols whenever it is necessary.

Thanks for raising this question, myself tried to verify this when I was reading Koenker's monograph Quantile Regression. In general, the expression he displayed was correct, but I think he didn't write everything clearly (at least lacking necessary explanations for some parts) so I would like to present my own derivation here and call for more discussion.

Given a multivariate-variable and real-valued function $f(x_1, \ldots, x_n)$, the directional derivative of $f$ along a direction $\mathbf{w}$ at $\textbf{x}$ is defined to be: $$\nabla_\mathbf{w}f(\mathbf{x}) = \lim_{h\to 0}\frac{f(\mathbf{x} + h\mathbf{w}) - f(\mathbf{x})}{h}$$ We shall verify the claimed result following the above definition. To alleviate the notation, denote $y_t - x'b$ by $u_t$, $\psi(b + hw; \theta, y, X)$ by $f(h)$, we want to show the limit of $(f(h) - f(0))/h$ exists and has the claimed form.

Straightforward algebra shows: $$f(h) - f(0) = \sum_{t = 1}^T\left[-(u_t - hx_t'w)\mathrm{I}\{u_t - hx_t'w < 0\} - hx_t'w\theta + u_t\mathrm{I}\{u_t < 0\}\right]$$ where $\mathrm{I}(\cdot)$ is the indicator function.

We may now evaluate the limit term-wisely, by discussing the positiveness of $u_t$:

If $u_t > 0$, then the summand is $-hx_t'w\theta$ (the indicator function always takes zero when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T-\theta x_t'w$$

If $u_t < 0$, then the summand is $(1 - \theta)hx_t'w$ (the indicator function always takes one when $h$ becomes sufficiently small), therefore: $$\lim_{h \to 0}\frac{f(h) - f(0)}{h} = \sum_{t = 1}^T(1 - \theta) x_t'w$$

For these two cases, the declared expression form were all successfully confirmed. However, the annoying and unclear case is that $u_t = 0$, for which, the summand is: $$(\mathrm{I}\{-hx_t'w < 0\} - \theta)hx_t'w$$ Let $g(h) = (\mathrm{I}\{-hx_t'w < 0\} - \theta)x_t'w$, namely, the function after the above expression divided by $h$. Clearly, $g(h)$ is not differentiable at $0$. Hence it is very irresponsible to just state the result without any further explanation. We may discuss the right/left limit of $g(h)$ as $h \to 0$, which depends on the value of $x_t'w$. $$\lim_{h \to 0^+} g(h) = (1/2 - 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w \\ \lim_{h \to 0^-} g(h) = (1/2 + 1/2\mathrm{sgn}(-x_t'w) - \theta)x_t'w$$

So the result reported by the author is essentially the "right directional derivative" instead of "directional derivative".

Answer 2

The objective function for quantile regression is a weighted sum of the absolute deviations, where the weights depend on the quartile $\tau$: $$R(\beta_\tau)=\sum_i^N \rho_\tau\cdot \vert y_i-x_i'\beta_\tau\vert=\sum_{y_i \ge x_i'\beta_\tau} \tau \cdot (y_i-x_i'\beta_\tau ) +\sum_{y_i<x_i'\beta_\tau} (\tau-1) \cdot ( y_i-x_i'\beta_\tau ).$$

The directional derivative of this puppy in direction $w$ is

$$\nabla R(\beta_\tau) = \frac{\partial \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (y_i-x_i'(\beta_\tau +tw) )}{\partial t} \big|_{t=0}+\frac{\partial \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot ( y_i-x_i'(\beta_\tau +tw))}{\partial t}\big|_{t=0}. $$ which is equal to

$$ \sum_{y_i \ge x_i'(\beta_\tau+tw)} \tau \cdot (-x_i'w) + \sum_{y_i<x_i'(\beta_\tau+tw)} (\tau-1) \cdot (-x_i'w), $$

which I believe is equivalent to what you have above.