I'm dealing with a dataset that comprises ~30 groups of observations. In each group, we measured independent and dependent variables a certain number of times. If I plot the relationship between independent and dependent variables for each group separately, I get something like this (showing just 4 of the groups here, with the independent variable on the X and dependent on the Y):
My ultimate goal is to determine whether the effect of the independent variable (i.e., the slope of a linear regression
dep ~ ind) is significantly different between groups, and to quantify which groups have the largest/smallest effects.
To do this, I've put every data point into a single dataframe, with a column indicating the group it came from. Then, I ran the following model, calculating an interaction term for
lm1 = lm(dep ~ ind * group, data=df)
To figure out whether a model that accounts for differences between groups is a better fit than a model that doesn't, I also fit the following model:
lm2 = lm(dep ~ ind, data=df)
And then I compared the two models using ANOVA:
The p-value of the ANOVA is about 1e-5, suggesting that the model that takes
group into account is the better fit.
However, I've also done the following to figure out if effects in particular groups are larger/smaller than in others:
lst <- lstrends(lm1, "group", var="ind")
Here, I'm performing pair-wise comparisons of effects between all groups, and after correcting for multiple testing, none of the pairwise comparisons are significantly different.
To me, this suggests that overall, a model that takes group into account is a better fit. But because the number of data points in each group is small (~8-10), the per-group regressions produce large confidence intervals, making it difficult to find significant differences between groups.
Am I thinking about this correctly?
(Apologies if a similar question has been asked before, I couldn't find any questions that directly addressed my concerns.)