I want to test whether an intervention affects a continuous outcome over 5 possible time points. How do I decide whether to include random slopes? I want slope to be able to vary between intervention group (by including a time * intervention interaction fixed effect) but not necessarily person to person within treatment group. From a theoretical standpoint, I would expect different people in the population to have different slopes for this outcome (anxiety) simply since there are so many factors that impact it, but I mostly care about the effect of the intervention. Intervention group was randomized. Is there a clear theory-based reason to go either way? Should I be using the (SPSS output) value of variance of the slope to help make that decision (farther from zero = random slopes more important)? I included a covariance significance test (TESTCOV in PRINT subcommand) in my output as well, and the random slopes are statistically significant in the Estimates of Covariance Parameters table, but I also read that many statisticians don't agree with the use of that test.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
8
There are two main considerations when choosing whether to specify random slopes for a variable:
Is it biologically / clinically / theoretically possible for each subject (or whatever the grouping variable is) to have their own slope with respect to that variable ? Obviously this also implies that the variable varies within level of the grouping variable.
Does the data support a model with random slopes ? That is, does the model converge normally ? Quite often the inclusion of random slopes leads to convergence problems, especially when correlations between the intercepts and slopes are also estimated.
I would refrain from any significance tests. If random slopes are justified and the model converges, retain the random slopes.
answered Sep 21, 2021 at 17:20
-
$\begingroup$ Thank you, this is helpful. Would you be able to tell me more about the first consideration? It seems like most person-level psychological outcomes could ostensibly have their own slope, or at least it's hard for me to think of outcomes where that would not be the case. Is there a level of hypothesized within-population slope variability that merits trying to include random slopes, or some other narrower criteria, or is the answer that Yes, typically when analyzing psychology data via growth curve modeling, one would indeed want to try to use random slopes? $\endgroup$– L.S.Commented Sep 23, 2021 at 20:41
-
$\begingroup$ Additionally, when you say to refrain from significance tests, are you also saying to steer clear of looking at the Covariance Parameters output in SPSS? My understanding is that the parameter estimates in this table tell us how important the random intercept, and random slope are, as well as how correlated the two are. $\endgroup$– L.S.Commented Sep 23, 2021 at 22:25
-
$\begingroup$ You're welcome. In such cases, then yes, I would seek to fit random slopes. As for significance tests - I am only talking about the use of p-values. The variance components themselves are certainly important theoretically, so if the model converges without error or warning - retain them. $\endgroup$ Commented Sep 24, 2021 at 7:40
-
$\begingroup$ Hi Robert, I'd like to ask a follow up question - It seems common in social and behavioral sciences to report on p values for random effects. What is the reason you recommend against this? Is it related to broader critiques and limitations around p values or are there concerns about significance tests that are specific to their applications for random effects? I have attempted to find an answer and have not been able to. If you could refer me to any literature or recommend any search terms that I could use to do independent research, I would appreciate it. (continued in next comment) $\endgroup$– L.S.Commented Nov 18, 2021 at 18:22
-
$\begingroup$ I read that random effects significance tests are considered questionable on the UCLA IDRE Statistical Consulting Web Resources page for SPSS MIXED and referred to the methods article they reference in making this claim (Parameter Estimation and Inference in the Linear Mixed Model by N. N. Gumedze and T. T. Dunne in Linear Algebra and its Applications, 2011). I consulted with a faculty statistician who was not familiar with the argument against random effects p values. They looked at the article as well, and concluded that the p values are not invalid. (continued in next comment) $\endgroup$– L.S.Commented Nov 18, 2021 at 18:25