# Model that takes grouping factor into account is a significantly better fit, but no significant differences between groups…unusual?

I'm dealing with a dataset that comprises ~30 groups of observations. In each group, we measured independent and dependent variables a certain number of times. If I plot the relationship between independent and dependent variables for each group separately, I get something like this (showing just 4 of the groups here, with the independent variable on the X and dependent on the Y):

My ultimate goal is to determine whether the effect of the independent variable (i.e., the slope of a linear regression dep ~ ind) is significantly different between groups, and to quantify which groups have the largest/smallest effects.

To do this, I've put every data point into a single dataframe, with a column indicating the group it came from. Then, I ran the following model, calculating an interaction term for group.

 lm1 = lm(dep ~ ind * group, data=df) 

To figure out whether a model that accounts for differences between groups is a better fit than a model that doesn't, I also fit the following model:

 lm2 = lm(dep ~ ind, data=df) 

And then I compared the two models using ANOVA:

 anova(lm1, lm2) 

The p-value of the ANOVA is about 1e-5, suggesting that the model that takes group into account is the better fit.

However, I've also done the following to figure out if effects in particular groups are larger/smaller than in others:

 library(lsmeans) lst <- lstrends(lm1, "group", var="ind") pairs(lst) 

Here, I'm performing pair-wise comparisons of effects between all groups, and after correcting for multiple testing, none of the pairwise comparisons are significantly different.

To me, this suggests that overall, a model that takes group into account is a better fit. But because the number of data points in each group is small (~8-10), the per-group regressions produce large confidence intervals, making it difficult to find significant differences between groups.

• I don't know what lsmeans does but why are you not looking at the lm1 output? The linear model will give you the p-values for all the tested hypotheses. – user2974951 Sep 17 '18 at 6:28