2
$\begingroup$

I have a dataset ($n=700$) where I measured $x$ and $y$ (continuous) at three points in time ($T_0$, $T_1$, $T_2$). I am interested in running a mixed model analysis in SPSS.

When I run normal linear regression analysis on my three separate datasets ($T_0$, $T_1$ and $T_2$) with $x_1$ and $y_1$, $x_2$ and $y_2$ and $x_3$ and $y_3$ separately, I get three highly significant results. However, when I run the mixed model analysis in my long (combined) data structure, $x$ is no longer significant.

Why does this happen?

$\endgroup$
2
  • 1
    $\begingroup$ Just to be clear, is the significance test you are performing for the mixed model a Type III sum of squares test? $\endgroup$
    – AdamO
    Oct 20, 2020 at 15:39
  • $\begingroup$ Hi Adam, thank you for your response. I am not sure what the default is in SPSS, I just run the Linear Mixed Model analysis and look at the p-value of my result.Restricted Maximum Likelihood is automatically selected, if that is of any help. $\endgroup$
    – Kate
    Oct 20, 2020 at 16:27

1 Answer 1

1
$\begingroup$

Assuming all things are setup correctly, there are 3 avenues of investigation.

Case 1: the trend of X and Y is non-homogeneous.

Explanation: the mixed model pools the separate cross-sectional analyses of time point 1, 2, and 3. However, if the trend is non-homogeneous, as evidenced by highly inconsistent cross-sectional slopes and estimates, the power of the mixed analysis is diminished.

Solution: provide forest plot of estimates and 95% CIS from cross sectional models. Consider using a GEE, or adjust for time and it's interaction with "X" in the mixed model. Note: the correlation structure of the mixed model should be exchangeable to prevent singularity.

Case 2: the correlation structure is misspecified

Explanation: In a panel design, there is correlation within participants and within similar time frames within a participant. Adjusting for "X" or time or other blocking factors may reduce the residual correlations so that a weaker correlation structure (even independence) is justified.

Solution: obtain estimates of intraclass correlation and plot variograms

Case 3: the intraclass correlation is too high

Explanation: it takes a little extra power to estimate the correlation structure. Consider that if three replications of a design were performed, and the repeated measures were perfectly correlated, the conservative assumption would be that the correlation was too high to obtain any additional precision from the repeated experiments, so they are effectively thrown away. Of course, the regression for time point 1 of X on Y was significant, so we have to consider that the "price" of fitting a more complex model has reduced the overall precision.

I may update the answer if more detail is provided.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for your help. About your third point: my ICC is 77% if correct, and I can see in my data that measurements don't really change over time. Does this mean that it's better to refrain from doing a longitudinal analysis? $\endgroup$
    – Kate
    Oct 20, 2020 at 16:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.