1
$\begingroup$

I am running a hierarchichal multiple linear regression with 4 steps containing theoretically justifyable variables:

Outcome: pain rating Step 1: demographic variables (age, gender) Step 2: Pain sensitivity measures (2 variables) Step 3: cognitive function (1 variable) Step 4: specific thoughts regarding pain (2 variables)

Here is my problem: when entering the cognitive function variable (calles Stop-Signal reaction time, lenient criteria) in Step 4, the change in R square is significant

model summary

Also the F-Test is significant:

ANOVA

However, in the final model, the standardized beta (first yellow column) of the Step 3 variable is not significant (second yellow column). This seems to be only the case after the variables of step 4 have been entered:

Regression coefficients

The variables of Step 4 are not correlated with the variable in Step 3:

Interkorrelation of predictors

Thus, it doesn't seem to be a problem of multicollinearity. I cannot make sense of this and I appreciate any help!

$\endgroup$
0
$\begingroup$

When moving from model 3 to 4, you are adding two variables instead of one. Not only are you adding $g + h$ but implicitly the interaction $g * h$. If there is no evidence of a correlation, there could potentially be an interaction effect between the two KRSS measures that moderates (i.e. attenuates) the effect of the Stop-Signal RT $(0.242 < 0.363)$ as well as the PPT baseline $(-0.397 < -0.215)$.

However, I would first isolate each of the KRSS measures and build two models, let's call them 4a and 4b. In 4a you add the KRSS Thought suppresion measure to Model 3, and in 4b you instead add KRSS HilfHoffnungslosigkeit. These two steps could rule out any individual moderation and validate your correlation analysis.

In case neither the models 4a or 4b individually moderate the RT or the PPT scores, it is likely a moderation-by-interaction effect.

To test this for the interaction $g * h$, compute a new variable: KRSS Thought Suppresion * KRSS HilfHoffnungslosigkeit and include this variable in the correlation analysis. (If the variables are of different scales, you should standardize them before the multiplication.)

Also note that gender seems to play a much lesser role in Model 4 than in 3: $(-0.1 << -0.013)$, suggesting perhaps that this effect was absorbed by one of the KRSS variables.

If you are interested in the effects of adding variables, a more detailed approach would be to extend the analysis with mediation[1] / moderation[2] analysis. That is also the basis for my recommendation, so the procedure is similar, but if you have a grasp of these concepts you can make a nice flow chart.

Finally, because the $F$-test for models 1 and 2 are not significant, you could exclude these variables from the final model. Naturally, you should still report the negative findings of the hierarchical analysis. You could also plot each model in a single plot and compare the slopes and the intercepts to get a sense of how the hierarchical steps affect the model. If you are using SPSS, use the Save function in the Regression window and then create a combined line plot of the $\hat y$ values for each model.

I hope this helps.

$\endgroup$
0
$\begingroup$

Thank you very much for your helpful suggestions!

Indeed, there are some interesting effects when looking at KRSS thought suppression and KRSS hilf/Hoffnungslosigkeit. For the sake of clarity, I will only report the results regarding KRSS thought suppression now.

It actually seems that Stop-Signal RT is not affected by thought suppression, as the beta weight nearly stays the same and is still significant. However, PPT_baseline (PPT_prestat_all_sites in this table) is nearly half after introducing thought suppression:

Regression coefficients model 4a

Indeed, there is a significant inverse correlation between thought suppression and PPT baseline (r= -.39, p=.012) which also makes sense in light of the theoretical background.

scatterplot baseline PPT and thought suppression

Would it be suitable to do a moderation analysis at this point, with baseline PPT as a moderator for the relationship between thought suppression and the outcome variable exercise-induced pain? Theoretically, that would make sense. However, the sample size appears to be too small to do that (N=40), and, as the coefficient table shows baseline PPT and the outcome are not correlated.

Thank you!

$\endgroup$
  • $\begingroup$ I would focus on partial correlations. If it's a new type of measurement, I would focus more on exploratory data analysis rather than confirmatory, even if that involves multiple tests. Add a third dimension, like color, to a two-dimensional plot to visualize more complex interactions. It is possible to dig deeper with regression diagnostics, but at this point I would try a MAGIC interpretation: Magnitude (e.g. the non-standardized betas) Articulation Generality Interestingness Credibility en.wikipedia.org/wiki/MAGIC_criteria $\endgroup$ – noumenal May 1 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.