0
$\begingroup$

I am running a moderated regression, N = 1515. I have one outcome variables, one moderator, one predictor and 5 controls. I am using SPSS for my analysis. I first performed linear regression to check for assumption violations. For this, I centered my predictor and outcome as they form the interaction term in my model. Then I checked for the usual assumptions of linear regression. My data is not normally distributed but given the large N, I ignored this violation. Mainly, my data also violated homoscedasticity assumption so I decided to run weighted least square regression WLS instead of OLS. For this, I took the absolute value of my residuals after running the OLS, used the absolute residuals and regressed them on all the variables in my model (centered predictors, interaction, controls) and them used the predicted values, used 1/(pred^2) to create the weights and then ran the WLS model. When I ran the OLS, my interaction term was not significant but in WLS it was significant. First, I want to know if this can happen or did I do something wrong? second, how do I now do the simple slope analysis on the WLS model ?

$\endgroup$

1 Answer 1

0
$\begingroup$

The ordinary least squares (OLS) and weighted least squares (WLS) models are different, so you shouldn't be surprised that they can give different results. The standard error of a regression coefficient (like that for your interaction/moderation term) is related to the residual error variance, so if you have an inappropriate estimate of that variance (as in OLS when there is heteroscedasticity) then the standard errors of regression coefficients will be correspondingly inappropriate.

That's the point of WLS: to account for the "known" (at least, approximately) sources of variance so that you get a better estimate of (ideally homoscedastic) residual variance. Then you also get better estimates of the standard errors of regression coefficients.

It looks like your method of estimating weights is consistent with recommendations like those on this Penn State web page, modeling standard deviations as a function of predictors (which places less weight on outliers than does modeling variance directly) and then using the inverse squares of the modeled standard deviations as observation weights. There doesn't seem to be anything obviously wrong in what you did. There's always some question when you have a multi-step analysis that uses the outcomes in an early step to adjust the analysis in a later step, but that's common practice in this situation. Just present what you did clearly.

You must consider, however, whether the "statistical significance" you found for the moderation is matched by practical significance. With such a large data set you run a risk that even a minute, inconsequential moderation effect will be found to be "statistically significant." In general, it's not a good idea to place too much importance on the typical arbitrary p < 0.05 cutoff for "significance." Apply your understanding of the subject matter to decide whether the moderation is important, not just "significant."

The "simple slopes" analysis should proceed as it normally would, except that you base the necessary predictions and standard errors on the WLS model instead of the original OLS model. Any questions about that specific to SPSS would be off-topic on this site, however.

$\endgroup$
2
  • $\begingroup$ thank you for your response. that's helpful. i just have a followup question, in one of the models, during WLS my residual standard error of the regression model is more than the OLS model. Does this indicate that OLS is a better fitting model? $\endgroup$
    – Tg51
    Jul 14, 2023 at 11:06
  • $\begingroup$ @Tg51 quite possibly the opposite. In OLS with all points weighted equally, an outlying point can lead to a model that fits that point better than a "true" model would. If you specified a high expected variance for that point it would be underweighted in WLS. The WLS fit might then deviate far from that point and lead to a large contribution to residual standard error. What's important is the pattern of studentized residuals that take the weights into account; see this page. $\endgroup$
    – EdM
    Jul 14, 2023 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.