0
$\begingroup$

I would like to compare the goodness of fit for a regression model fitted to two separate groups (patients and controls).

I want to compare goodness of fit (as opposed to difference in slope). I thought one way of doing this would be comparing the R value of the model in patients vs the value in controls in a Fisher R-to-z comparison.

My question is whether it is more appropriate to use the 'Multiple R-squared' or the 'Adjusted R-squared'. Additionally if the Adjusted R squared is preferred then how would one deal with negative values?

Or please do let me know if the whole approach is wrong headed

Thanks

Rob

$\endgroup$
  • $\begingroup$ What is your broader goal? $\endgroup$ – mkt - Reinstate Monica Feb 23 '18 at 11:45
  • $\begingroup$ To determine whether a relationship appears to be present in one group, but not in the other, and ascribe a degree of statistical significance to this $\endgroup$ – RobMcC Feb 23 '18 at 12:29
  • $\begingroup$ Note that showing that there is a relationship in one group and not in the other is not a basis for concluding that there is a difference between the two groups (doing so amounts to accepting the null for one of the groups). To answer your question, you need to directly test for an interaction. $\endgroup$ – dbwilson Mar 16 '18 at 16:02
  • $\begingroup$ Many thanks - I suggested performing an R-to-z in order to address this? $\endgroup$ – RobMcC Mar 17 '18 at 16:14
0
$\begingroup$

If you have exactly the same number of estimates, I believe both $R^2$ and adjusted $R^2$ would produce similar results. To my knowledge, the adjusted $R^2$ penalizes the inflation of the $R^2$ that comes from increasing the number of predictors. Given the two equations are equivalent, you should be fine by going with the regular $R^2$.

**** OLDER RESPONSE **** Sorry, I misunderstood the question. Disregard the response below:

The question is a bit vague. I'm assuming you are using R. Then,

    mod1 <- lm(y ~ x1 + x2 + x3, data=patients)
    mod2 <- lm(y ~ x1 + x2 + x3, data=controls)
    anova(mod1, mod2)

I hope this helps.

$\endgroup$
  • $\begingroup$ Could you explain how this answers the question about goodness of fit? $\endgroup$ – whuber Mar 16 '18 at 15:47
  • $\begingroup$ Maybe I misunderstood @RobMcC 's question, but it seems to me that he needs to know if the equations are different. It is very hard to evaluate without a more detailed example. $\endgroup$ – Iuri Gavronski Mar 16 '18 at 15:56
  • $\begingroup$ There's a more fundamental issue here concerning what an ANOVA of two models using two different sets of data even means, how to interpret it, and how it might be related to the question. I am trying to suggest that your answer looks problematic and requires further explanation, regardless of how you might understand the question. $\endgroup$ – whuber Mar 16 '18 at 16:37
  • $\begingroup$ Many thanks for your answer, but isn't the anova here effectively examining whether there is a difference in slope as opposed to a difference in goodness of fit? $\endgroup$ – RobMcC Mar 17 '18 at 16:13
  • $\begingroup$ @RobMcC: you normally test the difference of F using anova(). For example, if you had mod1 with x1 and mod2 with x2 and x3, the anova() would test the hypothesis that either x2 or x3 are significantly different from zero: mod1 <- lm(y ~ x1, data=p) mod2 <- lm(y ~ x1+x2+x3. data=p) $\endgroup$ – Iuri Gavronski Mar 17 '18 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.