2
$\begingroup$

Research context:

(1) What's the effect of a binary X1 (gender) on a continuous Y (index for consumption of femine-gendered goods), controlling for demographics. (2) Is this relationship moderated by continuous X2, X3, X4 (gender salience, perceived differences between genders, amount of close friends from other gender).

Model:

I ran a hierarchical regression model using Stata. Most effects are small, but still interpretable. The effect structure makes substantially sense and is in line with expectations.

Problem:

A reviewer suggested to use squared semi-partial correlations as alternative effect size. I computed those (using Stata's pcorr). I was baffled. The semi-partial correlations suggest that the predictors (especially the interactions) do not explain any variance in Y (- most of them well below 1%).

I’m aware of the general argument that small changes in R² can still make a big difference in real life. However, I ask myself, whether the differences between (my interpretation of) the regression coefficients and the semi-partials aren't too large to be able to follow this argument?

Questions:

How could I explain the different results from the semi-partials ("only 2% variance uniquely explained by gender") and the raw regression coefficients b (e.g., "changing gender increases outcome by almost a third of its theoretical scale“)?

How could I justify that I still stick to the interpretation of regression coefficients?

From here, I understood that the unsquared semi-partial correlations are very similar to standardized regression coefficients (but not able to exceed 1). From here I learned that semi-partials are often used in the decision about the inclusion of predictors. Different to these discussions, I'm seeking for help in regard to:

Which statistic I should rely on in the interpretation of the substantial findings - the explained variance in Y, or the change in Y when X changes one unit?

What type of research questions lends itself towards one or the other statistic?

How could I convince the reviewer that the effect size I choose is the most informative in my context?

Regression results and semi-partial correlations

$\endgroup$

1 Answer 1

1
$\begingroup$

The semipartial correlation of a predictor measures the (square root) of the decrease in R² when said predictor is removed from the full model.

If, for some reason, you have highly correlated predictors (e.g. independent variables, regressors, features, covariates), the semipartial correlation will often be small.

The most extreme example of this would be if the covariance of your predictors isn't full rank. In this case, you'd need to add regularization to fit the least-squares model, which will encourage the fit to distribute weights over redundant predictors.

In this case, any single predictor can be removed without reducing the information in your predictors, (or the R² in your model), so the semipartial/part correlation could show zero, even if each predictor is quite informative individually.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.