2
$\begingroup$

I've been looking for a method to calculate $R^2$ on a subset of the samples (a subset of the instances, not a subset of features), and found this answer from Dave. It suggests using the mean of the original samples ($\bar{y}$), rather than the mean of the subset, when calculating the TSS - .i.e.:

$R^2=\dfrac{\sum_j (y_j - \bar{y})^2 - \sum_j (y_j - \hat{y})^2}{\sum_j (y_j - \bar{y})^2}$.

Using this method has resolved the problem I'm having when using the subset mean to calculate $R^2$, where I get a low or negative value if my subset has very low variance (e.g. if the subset is instances with target values in a small range), and I'd like to use Dave's method in a paper I'm writing.

I've searched for academic references for this method of calculating $R^2$ but I have not found one so far.

Does anyone know of a suitable reference to use for this?

Improve this question
$\endgroup$
3
  • $\begingroup$ I don't have a reference for you, but if you articulate your argument for why my answer you linked is a reasonable approach, I do not believe that you need a reference to use my approach in your paper (and then we will all cite you when we want to use that approach). Do you understand my rationale in the answer you linked? $\endgroup$
    – Dave
    Mar 30 at 14:15
  • 1
    $\begingroup$ Thanks Dave. I'll do that if I can't find a reference. My understanding is that using your approach, I'm comparing the performance of my model over the sample subsets to the same baseline that I'm using when calculating $R^2$ for the full set of samples. $\endgroup$
    – Lynn
    Mar 31 at 12:27
  • $\begingroup$ That’s exactly how I se it! Would you be interested in posting a self-answer to close-out this question? $\endgroup$
    – Dave
    Jun 13 at 7:04

1 Answer 1

Reset to default
1
$\begingroup$

As my question is so far unanswered and I haven't been able to find a reference, I'm posting how I resolved this as an answer.

In my paper I justified this approach with the statement "When $R^2$ is calculated for a subset of samples, $\bar{y}$ is the mean of the measured LFMC for the full sample set, not the mean of the subset. By using this value for the mean, all $R^2$ calculations are compared to the same baseline, thus allowing comparisons between $R^2$ for different subsets of samples."

Improve this answer
$\endgroup$
1
  • $\begingroup$ This seems totally aligned with my comment. $\endgroup$
    – Dave
    Jul 1 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.