1
$\begingroup$

I need to see if different fits of different data sets are significantly different, preferrably in R.

I have sample series A, B and C. These are different in length and all contain variables n, m, o which in turn depend on variables x, y, z...

Fitting these different series to assess how they behave as we change x, y, z is easy enough, but how can I check if a fit assessing series A variable n is significantly different from one concerning series B or C, variable n?

Or to put it differently:

mydata=read.csv("Data.csv")
fitA=lm(Formula=n ~ x + y + z,subset=Group=="A",data=mydata)
fitB=lm(Formula=n ~ x + y + z,subset=Group=="B",data=mydata)
fitC=lm(Formula=n ~ x + y + z,subset=Group=="C",data=mydata)
sometest(fitA,fitB,fitC)

What can I use where I squeezed in sometest? Not necessarily using lm as the fitting method, but for the current data set it appears suitable, so we can assume it is for this question.

$\endgroup$
2
  • $\begingroup$ You refer to these as "series", are they time series data or cross-sectional? Is there any reason you can't fit a model that includes all 3? $\endgroup$ Jan 12, 2016 at 18:32
  • $\begingroup$ Cross sectional. I don't think fitting all three series into one function will help as I need to see if there are differences between the series. I need to see how samples in series A behave in response to changed x, y, z as compared to how samples in series B or C behaves in response to the same changes. The x, y, z variables are likely independent. My naïve assumption (I am no statistician. At all.) was to fit them using MLR or somesuch and compare the fits. $\endgroup$
    – The V
    Jan 12, 2016 at 20:21

1 Answer 1

1
$\begingroup$

If you're interested in comparing $R^2$ for different groups you can extract the $R^2$ using:

R_Sq_A <- summary(fitA)$r.squared
    R_Sq_B <- summary(fitB)$r.squared

These can then be compared using:

library(psych)
paired.r(sqrt(R_Sq_A), sqrt(R_Sq_B), NULL, 
         dim(mydata[which(Group=="A"),][1], dim(mydata[which(Group=="B"),][1])

You can replace the dim(mydata[which(Group=="A"),][1] with the size of Group A since you know what that is.

$\endgroup$
6
  • $\begingroup$ The OP's question doesn't seem to be about $R^2$, but if the coefficients associated with the variables change. $\endgroup$ Jan 12, 2016 at 20:37
  • $\begingroup$ It seems to be about how the collective change of 'x, y, and z' affects various outcome variables. That can be interpreted at R^2...maybe I'm misreading though. If it's about the different effect of, say, x on Outcome1 for the three different groups, the entire sample should be used with a categorical variable indicating group membership say G - this varaible should be included in the model with an interaction term G*x... $\endgroup$ Jan 12, 2016 at 20:51
  • $\begingroup$ This shows how to test of 'different slopes' between groups (in SPSS, but it's easily extended to R): stats.stackexchange.com/questions/141325/… $\endgroup$ Jan 12, 2016 at 20:53
  • $\begingroup$ R squared is not a definitive solution to my question, is it? I confess, it has been years since my last statistics course, so I freely admit to being fairly ignorant on this, but I recall that R2 has some pitfalls. I would expect a method for comparing different fits of the three situations would be more robust. $\endgroup$
    – The V
    Jan 13, 2016 at 13:52
  • $\begingroup$ It's not a definitive solution, no - just a quick and common one and one where 'significance' can be easily tested for - which was in your question; this is what my answer provides. Since you're fitting three different models, it looks like you're interested in how well x,y,z explain Outcome1 in Group 1 vs Group 2, say. R^2 will give you this. The 'pitfalls' of R^2 that you're remembering might have to do with comparing nested models: models with more independent variables will have a higher R^2...so adjusted R^2 is used, for instance. But that's not what the question said was of interest... $\endgroup$ Jan 13, 2016 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.