Skip to main content
4 of 6
added 2 characters in body
Dave
  • 67.1k
  • 7
  • 105
  • 305

Nonlinear quantile regression SSReg analogue

I have recently remembered that $SSTot = SSRes + SSReg$ fails to hold in the case of nonlinear regression.

$$ y_i-\bar{y} = (y_i - \hat{y_i} + \hat{y_i} - \bar{y}) = (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y}) $$

$$( y_i-\bar{y})^2 = \Big[ (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y}) \Big]^2 = (y_i - \hat{y_i})^2 + (\hat{y_i} - \bar{y})^2 + 2(y_i - \hat{y_i})(\hat{y_i} - \bar{y}) $$

$$ \sum_i ( y_i-\bar{y})^2 = \sum_i(y_i - \hat{y_i})^2 + \sum_i(\hat{y_i} - \bar{y})^2 + 2\sum_i\Big[ (y_i - \hat{y_i})(\hat{y_i} - \bar{y}) \Big]$$

Therefore, for a regression in general that predicts conditional expected value, $SSTot =$ $SSRes + SSReg + Other$. In the case of linear regression, the $Other$ term drops out due to orthogonality.

Consequently, I think that we can write the $R^2$ for nonlinear regression as:

$$R^2 =\dfrac{SSReg}{SSTot} = 1 - \dfrac{SSRes + Other}{SSTot}.$$

This becomes the regular $R^2$ when the regression is linear and $Other=0$. When the regression is nonlinear, we still have a calculation that R or Python can handle just fine.

However, I want something akin to $R^2$ for quantile regression. I have reasoned through a way to get something for the case of a linear quantile regression, which is what Koenker and Machado use in their 1999 JASA paper, according to another post on CrossValidated. (I can't access the paper, but this equation makes sense to me.) Let $q$ be the quantile being estimated by quantile regression.

$$R^2_q = 1 - \frac{\sum_{y_i \ge \hat{y_i}} q \vert y_i - \hat{y_i} \vert + \sum_{y_i < \hat{y_i}} (1-q) \vert y_i - \hat{y_i} \vert}{\sum_{y_i \ge \bar{y}} q \vert y_i - \bar{y} \vert + \sum_{y_i < \bar{y}} (1-q) \vert y_i - \bar{y} \vert} $$

This uses the loss function for the model divided by the loss function for the naive model that takes the pooled $q^{th}$ quantile as the estimate, analogous to how $R^2$ divides the loss function of the model by the loss function of the naive model that takes the estimate as the pooled mean.

This assumes that the analogous $Other$ term is zero for the linear quantile regression, bringing me to my question.

What is that $Other$ component for a nonlinear quantile regression?

Proceeding analogous to my first set of equations for the regular regression case, I start with $\vert y_i - \bar{y} \vert = \vert y_i - \hat{y_i} + \hat{y_i} - \bar{y} \vert $. Breaking this into something related to $\vert y_i - \hat{y_i} \vert$ and $\vert \hat{y_i} - \bar{y} \vert$ is eluding me.

Dave
  • 67.1k
  • 7
  • 105
  • 305