22
$\begingroup$

Having included an quantile regression model in a paper, the reviewers want me to include adjusted $R^2$ in the paper. I have calculated the pseudo-$R^2$s (from Koenker and Machado's 1999 JASA paper) for the three quantiles of interest for my study.

However, I have never heard of an adjusted $R^2$ for quantile regression and wouldn't know how to calculate it. I am asking you for either of the following:

  • preferably: a formula or approach on how to meaningfully calculate an adjusted $R^2$ for quantile regression.

  • alternatively: convincing arguments to provide to the reviewers as to why there isn't such a thing as an adjusted $R^2$ in quantile regression.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Maybe cross validation? $\endgroup$
    – Christoph Hanck
    Commented Jul 4, 2016 at 17:29
  • 2
    $\begingroup$ I am not sure, else I would have given a proper answer... As the question seemed to generate much interest without generating answers, I wanted to get the discussion going. But broadly, adjusted $R^2$ suggests some kind of model selection is the goal, so CV seems like a default strategy that often works even when no specific formulae relying on specific likelihoods (like AIC) are available. It is however not clear (to me) why the reviewers want an adjusted $R^2$ in the first place. $\endgroup$
    – Christoph Hanck
    Commented Jul 13, 2016 at 7:12
  • 2
    $\begingroup$ This question has already been asked, discussed, answered, and it has a highly voted answer: stats.stackexchange.com/q/129200 Please, make this answer a comment. $\endgroup$
    – Toka
    Commented Jul 13, 2016 at 21:58
  • 3
    $\begingroup$ @Toka Thank you for finding the reference. Although it is a highly relevant discussion, I do not see anything in it that addresses the particular concern here, which is an adjusted (pseudo-)$R^2$ to be used as an analog of the adjusted $R^2$ of least-squares multiple regression. $\endgroup$
    – whuber ♦
    Commented Jul 13, 2016 at 22:32
  • 1
    $\begingroup$ In my opinion "I don't know how to calculate that. How do I calculate that?" is a perfectly good response to a reviewer request like that. Reviewers are not entitled to require analyses that they don't know how to do themselves, as I suspect is the case here. $\endgroup$
    – Kodiologist
    Commented Jul 20, 2016 at 17:56

2 Answers 2

Reset to default
5
$\begingroup$

I think what the reviewers are asking is to take the pseudo-$R^2$-values and "unbias" them for number of samples in the quantile range, $n_Q$, and number of parameters in the model, $p$. In other words, adjusted-$R^2$ in its usual context. That is that the corrected unexplained fraction is larger than the gross unexplained fraction by a factor of $\frac{n_Q-1}{n_Q-p-1}$, i.e.,

$1-R^{2*}=\frac{n_Q-1}{n_Q-p-1}(1-R^2)$, or, $R^{2*}=1-\frac{n_Q-1}{n_Q-p-1}(1-R^2)$

I agree with you about taking things too far, because this is already a pseudo-$R^2$-value and an adjusted-pseudo-$R^2$-value might leave the reader with an impression of performing a pseudo-adjustment.

One alternative is to do the calculations and show the reviewers what the results are and NOT include them in the paper, by explaining that it goes beyond what the published methods are that you are using and you do not want the responsibility for inventing an otherwise unpublished adjusted-pseudo-$R^2$ procedure. However, you should realize that the reason that the reviewers are asking is because they want assurances that they are not seeing gibberish numbers. Now, if you can think of another way of doing exactly that, assuring the reviewer(s) that the results are reliable, then the problem should go away...

One alternative is to include more references or information about the pseudo-$R^2$ values you are using, especially if you can show robustness, or precision. For example A Lack-of-Fit Test for Quantile Regression. Are the pseudo-$R^2$ values essential to the paper, or are there other ways to accomplish the same goal?

Sometimes, just deleting the problem is the simplest thing to do. Yes, we agree with you, exalted reviewer, your majestic infallibility is worshiped, $<$grovel, grovel$>$ problem deleted.

Improve this answer
$\endgroup$
0
$\begingroup$

You had better not use $R^2$ to compare two quantile regression models, because the quantile regression model's loss function is not based on MSE.

You can try AIC or BIC.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Hi, and welcome to the site. Could you expand a bit on the rationale behind your first sentence? $\endgroup$
    – Stephan Kolassa
    Commented Jul 13, 2016 at 6:41
  • 7
    $\begingroup$ I suspect there may be a "not" missing at the beginning. $\endgroup$
    – whuber ♦
    Commented Jul 19, 2016 at 13:56
  • $\begingroup$ (My edit was trivial and not an attempt to resolve @whuber's valid doubt -- that is for the OP.) $\endgroup$
    – Nick Cox
    Commented Aug 24, 2017 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.